-- Logs begin at Sun 2022-07-31 17:56:22 UTC, end at Sun 2022-07-31 18:58:14 UTC. -- Jul 31 18:23:00.154293 dv-cinder-849846-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 31 18:23:00.161945 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 31 18:23:00 2022] *** Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: nodename: dv-cinder-849846-5-iscsi Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: machine: x86_64 Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: clock source: unix Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: pcre jit disabled Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: detected number of CPU cores: 4 Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: current working directory: / Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: detected binary path: /usr/bin/uwsgi-core Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: your processes number limit is 47680 Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: your memory page size is 4096 bytes Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: detected max file descriptor number: 2048 Jul 31 18:23:00.163968 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: lock engine: pthread robust mutexes Jul 31 18:23:00.164864 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: thunder lock: enabled Jul 31 18:23:00.164864 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 31 18:23:00.164864 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 31 18:23:00.204003 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: Python main interpreter initialized at 0x556470f559c0 Jul 31 18:23:00.204003 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: python threads support enabled Jul 31 18:23:00.204003 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: your server socket listen backlog is limited to 100 connections Jul 31 18:23:00.204003 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: your mercy for graceful operations on workers is 90 seconds Jul 31 18:23:00.204394 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: mapped 403077 bytes (393 KB) for 2 cores Jul 31 18:23:00.204456 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: *** Operational MODE: preforking *** Jul 31 18:23:00.204750 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: *** uWSGI is running in multiple interpreter mode *** Jul 31 18:23:00.204750 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: spawned uWSGI master process (pid: 80187) Jul 31 18:23:00.207569 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: spawned uWSGI worker 1 (pid: 80202, cores: 1) Jul 31 18:23:00.207569 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: spawned uWSGI worker 2 (pid: 80203, cores: 1) Jul 31 18:23:00.207569 dv-cinder-849846-5-iscsi devstack@keystone.service[80187]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 31 18:23:00.204971 dv-cinder-849846-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 31 18:23:05.639244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 18:23:05.639244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes Jul 31 18:23:05.640039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 18:23:05.640039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: from cryptography.utils import int_from_bytes Jul 31 18:23:05.643077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 18:23:05.643077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Jul 31 18:23:05.643582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 18:23:05.643582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Jul 31 18:23:05.692059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 18:23:05.692259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 18:23:05.692431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] command line args: [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 18:23:05.692692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 18:23:05.692848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 18:23:05.692933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ================================================================================ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 18:23:05.692994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 18:23:05.693129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.693211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] command line args: [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 18:23:05.693370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_dir = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.693459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 18:23:05.693583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.693721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ================================================================================ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 18:23:05.693810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_source = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.693877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.693940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] debug = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_dir = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] config_source = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] debug = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.694971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_config_append = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_dir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.695986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_config_append = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_dir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.696990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.697922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.698951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.699977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] publish_errors = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] publish_errors = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.700983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] standard_threads = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.701885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] standard_threads = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_journal = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_json = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_stderr = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.702935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_journal = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_syslog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_json = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_stderr = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.703605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] use_syslog = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.703829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 18:23:05.703937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.704870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.705124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.705301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.705476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.705659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.705834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.706924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.707987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.708997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.709938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.710936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.711930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.712968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.713941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.714949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.715982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.716973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.717996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.718984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.719898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.720913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.721901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.722913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.723917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.724890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.725943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.726950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.727963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.external = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.external = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.728946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.password = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.token = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.password = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.729895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] auth.token = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.730904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.731896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732352 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.732923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733595 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.733932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.734951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.735943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.736926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.737935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.738903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.739942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.740931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.741927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.742927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.743922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744595 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.744954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.745951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.746965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.747968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.748887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.749925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.750959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.751988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.752971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.753967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.754981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.755961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.756070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.756536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.756658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.756786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.756899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.757898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.758908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.759942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.760917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.761917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.762920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.763901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.764915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.765927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.766937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.767997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.768956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.769988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.770994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.771895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.772933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.773943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.774943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.775944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 18:23:05.776054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.776054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.776054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.776054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.776461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.776572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 18:23:05.776680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80202) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 18:23:05.804696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.804951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.805175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.805263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.805480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.805700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.813531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.814274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.843413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.843618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.850348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.850536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.895771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.900095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.945543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.945691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.945813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:05.945895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.010996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.011100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.011193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.034962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.035099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.035192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.035315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.035390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.035452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.035524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.046448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.046575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.059912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.060678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80202) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 18:23:06.077655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.077840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 18:23:06.078105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.078232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.078362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 18:23:06.078452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 18:23:06.078614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.078700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.078818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 18:23:06.078899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.079122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.079263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.079344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.079406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.079624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.079880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.089056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.089331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.089598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.089724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.089916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.090045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.090194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.090413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.096863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.097489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.101586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.101860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.102139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.102263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.102399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.102480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.102684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.102883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.102970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.103995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.104133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.104221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.104295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.104539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.104797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.127201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.127922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.128681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.128982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.129436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.129717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.133963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.134240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.134513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.134713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.134985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.135263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.141107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.141393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.141912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.142190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.145553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.145891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.146110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.146397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.146679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.146950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.147218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.147496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.150396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.150705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.150988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.151270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.151543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.151832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.152099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.152359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.164446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.164722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.168623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.168747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.168906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.171995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.172287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.172853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.176048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.176348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.176719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.180253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.180747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.183683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.183912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.184166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.184625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.184950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.185229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.185518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.187151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.187443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.187722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.188002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.188280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.188563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.203940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.204221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.204490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.204677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.204805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.205011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.205278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.205542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.211654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.212465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.213220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.213466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.214044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.214311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.216637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.216637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.216916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.217152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.217305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.217424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.217522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.217716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.218000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.218286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.229741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.229870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.230030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.230948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.231082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.231163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.231353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.231478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.246935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.247168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.249928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.250193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.252952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.253282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.255522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.255863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.257428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.257696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.257819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.257954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.258033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.258280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.264447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.264782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.266060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.266395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.269073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.269359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.269482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.269609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.269691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.269895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.270033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.270245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.277934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.278216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.278337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.278466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.278568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.278872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.285382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.285663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.285786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80202) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 18:23:06.285933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.286976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.287979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.288957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.289081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.289336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80202) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 18:23:06.322221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556470f559c0 pid: 80203 (default app) Jul 31 18:23:06.322744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556470f559c0 pid: 80202 (default app) Jul 31 18:23:06.325574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d2e41-6538-4bed-a40c-42709a45aee8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:06.325857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d2e41-6538-4bed-a40c-42709a45aee8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:06.326125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d2e41-6538-4bed-a40c-42709a45aee8 None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:06.330619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Sun Jul 31 18:23:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:13.953804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:13.954425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:13.954425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:13.958917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:13.965298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:13.966065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:13.966181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:14.044276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: INFO dbcounter [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] Registered counter for database keystone Jul 31 18:23:14.069760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG oslo_db.sqlalchemy.engines [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80203) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 18:23:14.079170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writer thread running {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 18:23:14.266685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80203) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 31 18:23:14.267092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80203) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 31 18:23:14.321400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:14.323007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:14.470636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 31 18:23:14.650781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:14.729472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: INFO dbcounter [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] Registered counter for database keystone Jul 31 18:23:14.754150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG oslo_db.sqlalchemy.engines [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80202) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 18:23:14.766534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writer thread running {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 18:23:14.885185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cdfd7-cf3e-481c-b731-f34f75062230 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:14.890077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:14.890716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:14.890990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:14.891257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:14.900210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:14.951857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 31 18:23:14.952121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400919-2603-41cc-b6b1-46d02113f008 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:14.962530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Sun Jul 31 18:23:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:23:14.972011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:14.986639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a46995-bf6d-4f45-a123-15e2481bdfa3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:14.992417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:14.993598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:14.993875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:14.994146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] PATH_INFO: `/v3/projects/57e06c47f85e41698c0742550f1a32ec` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:14.995786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] RBAC: Authorizing `identity:get_project(project_id=57e06c47f85e41698c0742550f1a32ec)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:15.046322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING keystone.common.rbac_enforcer.enforcer [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 31 18:23:15.046659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa621f-7de0-4781-9976-5cdb23268ce6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:15.051410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Sun Jul 31 18:23:14 2022] GET /identity/v3/projects/57e06c47f85e41698c0742550f1a32ec => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:17.045591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf9852-d79e-4b9a-b793-8da893f464af None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:17.046169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf9852-d79e-4b9a-b793-8da893f464af None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:17.046169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf9852-d79e-4b9a-b793-8da893f464af None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:17.051339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:17.057120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:17.057577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:17.057661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:17.078913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:17.080534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:17.089829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 31 18:23:17.111364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf9852-d79e-4b9a-b793-8da893f464af None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:17.125741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf9852-d79e-4b9a-b793-8da893f464af None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:17.130565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88734803-8c50-48e1-8e8c-88563bdde0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:17.131510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88734803-8c50-48e1-8e8c-88563bdde0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:17.131789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88734803-8c50-48e1-8e8c-88563bdde0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:17.132057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88734803-8c50-48e1-8e8c-88563bdde0e3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:17.139879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88734803-8c50-48e1-8e8c-88563bdde0e3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:17.141688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88734803-8c50-48e1-8e8c-88563bdde0e3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:17.150703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Sun Jul 31 18:23:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:23:17.156186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:17.166282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f472b8-fbf1-47c7-9cb8-e7d5d6095dc0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:17.172026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3568b1-f190-493e-be8d-f0c7013dbc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:17.172658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3568b1-f190-493e-be8d-f0c7013dbc15 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:17.172965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3568b1-f190-493e-be8d-f0c7013dbc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:17.173233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3568b1-f190-493e-be8d-f0c7013dbc15 admin admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:17.174706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3568b1-f190-493e-be8d-f0c7013dbc15 admin admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:17.176592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3568b1-f190-493e-be8d-f0c7013dbc15 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:17.178154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Sun Jul 31 18:23:17 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:19.215292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b07c3-1dc3-48a5-b20d-782d3ba5fead None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.215878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b07c3-1dc3-48a5-b20d-782d3ba5fead None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.215878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b07c3-1dc3-48a5-b20d-782d3ba5fead None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.217483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:19.222672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.222936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac4806-ada7-459b-9802-ff49b0f5f6fb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.223305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.223305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.223432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac4806-ada7-459b-9802-ff49b0f5f6fb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.223496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac4806-ada7-459b-9802-ff49b0f5f6fb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.224812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:19.229618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.229899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.230176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.249438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:19.250604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:19.266713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80202) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 31 18:23:19.267128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG passlib.handlers.bcrypt [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80202) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 31 18:23:19.312034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:19.313658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:19.329246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 7/12] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:19.350976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:19.375407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:19.414971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 6/13] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:19.437511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:19.456551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2ee8-97ee-424d-b772-657f886fa05f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:19.490766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:19.491321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.491597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.491863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.536675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d1bb8-bb7c-4062-aec9-4a30559d2e86 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:19.539013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:19.559491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:19.561010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:19.574003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:19.575131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.575412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.575680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.584367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:19.588357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:19.597079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:19.598066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:19.598907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:23:19.604206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:19.664891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 8/15] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:19.670462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:19.720488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e52db0-4d64-407d-91d3-d1269248c7c9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:19.753137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:19.753672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.753946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.754214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.754937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:19.755236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:19.757086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:19.770086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26ca5-2d93-4769-b1b8-8b939bbdfb62 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:19.775283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 8/16] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:19.780917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:19.795582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:19.808498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:19.809178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.809441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.809719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.809882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:19.819895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:19.821403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:23:19.839797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:19.840941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:19.841871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:23:19.841871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:23:19.841871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:23:19.841871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:19.841871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:23:19.846983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:19.884309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9903c57-9652-459b-a07a-06d65d9494cf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:19.917223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:19.917896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:19.918164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:19.918429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:19.919972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:19.921448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:19.923062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:19.931195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:20.034692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:20.043139 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405eb6-4a46-4e26-bc1c-3a808633e40e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:20.063355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e3541-4744-4ae4-80e4-7a30d9950643 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:20.080869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:20.081514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:20.081790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:20.082055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:20.083253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:20.084717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:20.086318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:20.092244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:20.096263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:20.096912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:20.097187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:20.097451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:20.098138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:20.098432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:20.099805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4de511-6a9c-4770-8235-7ab06b2453bf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:20.118625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:20.205619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3cb82-a12c-4d74-889e-24410efd5027 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:20.249089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:20.249753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:20.249986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:20.250254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:20.251062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.domain.id=default, user.id=a62909c025a448a2a69b126becb412ba) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:20.251366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:20.253913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaad29c-92c4-457c-ac23-828b4813502c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:20.267154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Sun Jul 31 18:23:20 2022] GET /identity/v3/role_assignments?user.id=a62909c025a448a2a69b126becb412ba&scope.domain.id=default&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 233 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:22.569768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f94dde-9d7d-4554-b4f1-2c1b413c4522 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:22.570703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f94dde-9d7d-4554-b4f1-2c1b413c4522 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:22.571163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f94dde-9d7d-4554-b4f1-2c1b413c4522 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:22.572711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:22.577999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70193762-92ce-4767-afe2-43fc40ad53b2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:22.578791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70193762-92ce-4767-afe2-43fc40ad53b2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:22.579248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70193762-92ce-4767-afe2-43fc40ad53b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:22.603355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-70193762-92ce-4767-afe2-43fc40ad53b2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:22.604926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-70193762-92ce-4767-afe2-43fc40ad53b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:22.632148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:22.659766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96ae13-fe76-4bd0-9520-9ff7e47e7935 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:22.660070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96ae13-fe76-4bd0-9520-9ff7e47e7935 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:22.660308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96ae13-fe76-4bd0-9520-9ff7e47e7935 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:22.661677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 12/23] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:22.667103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:22.667392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:22.667661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:22.700846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:22.702051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:22.726671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 12/24] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:22.761760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fd533-47ba-4f82-8c86-fa7af802d534 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:22.762064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fd533-47ba-4f82-8c86-fa7af802d534 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:22.762297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fd533-47ba-4f82-8c86-fa7af802d534 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:22.763649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 13/25] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:22.765549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:22.768984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:22.769269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:22.769530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:22.799967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:22.801200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:22.829053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 13/26] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:22.829974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:22.926587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 14/27] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:22.928153 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:22.962662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71872-b992-4a83-a017-e1233ecdef9d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.003036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.004105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.004570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.005131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.019239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.021118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f0b1662d-19d2-4d85-b910-9f0476e8379c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:23:23.040286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:23.041243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:23.042236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:23:23.042236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:23:23.042236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:23:23.042236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:23.044001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 14/28] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:23.045737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153de56d-4a24-40c5-a948-53a2e446471a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.046238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153de56d-4a24-40c5-a948-53a2e446471a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.046678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153de56d-4a24-40c5-a948-53a2e446471a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.048605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 15/29] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:23.049930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153de56d-4a24-40c5-a948-53a2e446471a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:23.077464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:23.158589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:23.175669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7badd-d1d8-4e97-86da-32b8bdc0c7f7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.190543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153de56d-4a24-40c5-a948-53a2e446471a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.212014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.213020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.213504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.213960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.225073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:23.231061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:23.231923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:23.232663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:23.239300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 15/30] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:23.239300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:23.239995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a299d-ea3b-485e-969d-11a6b9ce1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.240489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a299d-ea3b-485e-969d-11a6b9ce1673 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.240785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a299d-ea3b-485e-969d-11a6b9ce1673 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.241051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a299d-ea3b-485e-969d-11a6b9ce1673 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.241781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a299d-ea3b-485e-969d-11a6b9ce1673 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.243169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a299d-ea3b-485e-969d-11a6b9ce1673 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.353677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de06545-50ff-4429-8757-dcd9629a26e3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.369095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 16/31] 10.222.0.49 () {60 vars in 1131 bytes} [Sun Jul 31 18:23:23 2022] POST /identity/v3/roles => generated 219 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:23.371002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.371486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.371941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.402450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:23.406649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:23.414440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.416513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.420956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.421605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.422564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:23.423053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.426029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.471393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 16/32] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:23.473955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:23.492918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:23.546578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 17/33] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:23.553933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:23.563861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:23.645386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1eeef-1466-4483-9a68-272d168754ac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.686268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.687239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.687718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.687895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082da8e-7e82-4269-b036-60fa9bcdfd80 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.688339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.689209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:23.689717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.691561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.703007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 17/34] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:23.704068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:23.726643 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a9548-486e-459b-9d22-37bd8810cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.727284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9548-486e-459b-9d22-37bd8810cf9f None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.727561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9548-486e-459b-9d22-37bd8810cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.727833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9548-486e-459b-9d22-37bd8810cf9f None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.728515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a9548-486e-459b-9d22-37bd8810cf9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.730142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a9548-486e-459b-9d22-37bd8810cf9f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.761413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 18/35] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:23:23 2022] POST /identity/v3/projects => generated 301 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:23.762753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff7bf-6b45-4fac-b6f8-2ef352fb49ce None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.763043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff7bf-6b45-4fac-b6f8-2ef352fb49ce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.763318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff7bf-6b45-4fac-b6f8-2ef352fb49ce None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.764547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 19/36] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:23.770041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff7bf-6b45-4fac-b6f8-2ef352fb49ce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:23.791369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:23.835882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8a736-d875-4f6f-a436-42aa7be215e8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.869199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:23.874039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.874979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.875410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.875830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.877107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.878868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.891814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff7bf-6b45-4fac-b6f8-2ef352fb49ce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:23.931908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34259d-f751-4554-a348-540869f9123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:23.932563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34259d-f751-4554-a348-540869f9123e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.932867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34259d-f751-4554-a348-540869f9123e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.933143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34259d-f751-4554-a348-540869f9123e None admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.935093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34259d-f751-4554-a348-540869f9123e None admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:23.936623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34259d-f751-4554-a348-540869f9123e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:23.939020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 20/37] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:23 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:23.940212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:23.940495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:23.940806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:23.961455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:23.962979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:24.016688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 18/38] 10.222.0.49 () {60 vars in 1131 bytes} [Sun Jul 31 18:23:23 2022] POST /identity/v3/roles => generated 225 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:24.022891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:24.045474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 21/39] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:24.068011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:24.130167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:24.157364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde81aee-34e5-4df6-a8a0-ba3435b88712 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:24.187216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2efd-74c4-4037-bb07-55172a787524 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:24.194043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:24.205379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725b60c-446f-42ea-a8f2-af12dfba6629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:24.206557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725b60c-446f-42ea-a8f2-af12dfba6629 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.207004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725b60c-446f-42ea-a8f2-af12dfba6629 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.207284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725b60c-446f-42ea-a8f2-af12dfba6629 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.208594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725b60c-446f-42ea-a8f2-af12dfba6629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:24.210447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725b60c-446f-42ea-a8f2-af12dfba6629 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:24.211777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 19/40] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:24.213147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898464a-db21-4e8e-805d-5561b6cc7e42 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.213430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898464a-db21-4e8e-805d-5561b6cc7e42 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.213699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898464a-db21-4e8e-805d-5561b6cc7e42 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.215067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 20/41] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:24.221828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898464a-db21-4e8e-805d-5561b6cc7e42 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:24.235449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288df2b-1a4f-43ea-8a3a-7b9e6d4c6d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:24.236622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288df2b-1a4f-43ea-8a3a-7b9e6d4c6d4c None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.236923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288df2b-1a4f-43ea-8a3a-7b9e6d4c6d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.237315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288df2b-1a4f-43ea-8a3a-7b9e6d4c6d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.238060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288df2b-1a4f-43ea-8a3a-7b9e6d4c6d4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:24.239453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288df2b-1a4f-43ea-8a3a-7b9e6d4c6d4c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:24.255488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 22/42] 10.222.0.49 () {60 vars in 1131 bytes} [Sun Jul 31 18:23:24 2022] POST /identity/v3/roles => generated 223 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:24.257314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.257625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.257893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.276364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:24.277664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:24.360835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 23/43] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:24.362947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898464a-db21-4e8e-805d-5561b6cc7e42 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:24.380297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:24.401659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:24.402355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.402637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.402913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] PATH_INFO: `/v3/domains/default/users/a62909c025a448a2a69b126becb412ba/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.405038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a62909c025a448a2a69b126becb412ba, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:24.406507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:24.425646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 21/44] 10.222.0.49 () {58 vars in 1361 bytes} [Sun Jul 31 18:23:24 2022] PUT /identity/v3/domains/default/users/a62909c025a448a2a69b126becb412ba/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:24.432661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:24.500177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90041fb-81d7-47bb-a411-7791c005d7fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:24.540608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14dad0-696d-44f8-a0b1-0d2834de5dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:24.541310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14dad0-696d-44f8-a0b1-0d2834de5dae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.541588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14dad0-696d-44f8-a0b1-0d2834de5dae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.541859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14dad0-696d-44f8-a0b1-0d2834de5dae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.543108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14dad0-696d-44f8-a0b1-0d2834de5dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:24.544594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14dad0-696d-44f8-a0b1-0d2834de5dae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:24.546276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 24/45] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:24.552441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:24.614529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:24.659339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86cae1-ac0b-4306-a2b8-33d33d51ebaa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:24.694855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72aef3-0e22-4b4f-8e93-10425c61d5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:24.695478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72aef3-0e22-4b4f-8e93-10425c61d5a4 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:24.695760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72aef3-0e22-4b4f-8e93-10425c61d5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:24.696032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72aef3-0e22-4b4f-8e93-10425c61d5a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:24.696718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72aef3-0e22-4b4f-8e93-10425c61d5a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:24.698219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72aef3-0e22-4b4f-8e93-10425c61d5a4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:24.734608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 22/46] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:23:24 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:26.400587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d87e3d-e7b7-4de2-9aea-af0e28341154 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.401153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d87e3d-e7b7-4de2-9aea-af0e28341154 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.401153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d87e3d-e7b7-4de2-9aea-af0e28341154 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.402514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 25/47] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:26.407992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.408436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.408594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.427392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:26.428930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:26.497178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 23/48] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:26.520109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d87e3d-e7b7-4de2-9aea-af0e28341154 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:26.622703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d87e3d-e7b7-4de2-9aea-af0e28341154 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:26.637237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:26.659028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b1556-a012-40d4-a7e5-0e324b761429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:26.659674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b1556-a012-40d4-a7e5-0e324b761429 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.659948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b1556-a012-40d4-a7e5-0e324b761429 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.660216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b1556-a012-40d4-a7e5-0e324b761429 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.671262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b1556-a012-40d4-a7e5-0e324b761429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e25b1556-a012-40d4-a7e5-0e324b761429 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:26.677881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:26.678703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:26.679548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 26/49] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:26.685095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:26.705886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0689b2-a4eb-4b60-92ef-add937c96a79 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.706165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0689b2-a4eb-4b60-92ef-add937c96a79 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.706437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0689b2-a4eb-4b60-92ef-add937c96a79 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.707773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 27/50] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:26.712528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.712823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.713100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.725364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:26.731107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:26.732368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:26.786596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa5360-3013-474a-b87e-273676ba6889 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:26.798143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 28/51] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:26.818783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:26.819398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.819748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.820020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.820687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:26.820912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:26.821052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:26.822431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:26.830601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 24/52] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:26.836073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:26.924998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d5fe8-6194-4fe0-8389-13b517d6edb8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:26.927222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:26.934724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:26.937727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0dde8-cf1c-4003-9565-09afa1fc888d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:26.959478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:26.960122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.960394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.960658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.961844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:26.963308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:26.964927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 29/53] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:26.970359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:26.970594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:26.970998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:26.971274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:26.971544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:26.972704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:26.974836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:26.975747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 25/54] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:26 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:26.980867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:27.070548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60858071-04e7-47b6-aecc-66c3fb8118e6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:27.078213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434bb10-19b1-4069-be0d-1296d1b0da2f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:27.102994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81b8f2-3b7c-41c4-ac9b-6e7f8906731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:27.103621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81b8f2-3b7c-41c4-ac9b-6e7f8906731a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:27.103899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81b8f2-3b7c-41c4-ac9b-6e7f8906731a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:27.104174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81b8f2-3b7c-41c4-ac9b-6e7f8906731a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:27.104888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81b8f2-3b7c-41c4-ac9b-6e7f8906731a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:27.106340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81b8f2-3b7c-41c4-ac9b-6e7f8906731a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:27.110861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:27.111496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:27.111774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:27.112042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:27.113277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:27.114700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:27.116320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 26/55] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:27.122382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:27.133039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 30/56] 10.222.0.49 () {60 vars in 1140 bytes} [Sun Jul 31 18:23:26 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:27.160255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:27.234750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c26e3-bf43-4f49-8353-66bfeb39aa15 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:27.271426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:27.272054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:27.272332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:27.272602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:27.273447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.domain.id=default, user.id=a62909c025a448a2a69b126becb412ba) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:27.273751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:27.275193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e788e-27cd-4f9f-b8b7-38e9d7a0ae05 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:27.282015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 27/57] 10.222.0.49 () {56 vars in 1322 bytes} [Sun Jul 31 18:23:27 2022] GET /identity/v3/role_assignments?user.id=a62909c025a448a2a69b126becb412ba&scope.domain.id=default&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:29.125909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899a27a-513c-4f57-945b-3b4bd3177233 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:29.126527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899a27a-513c-4f57-945b-3b4bd3177233 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:29.126527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899a27a-513c-4f57-945b-3b4bd3177233 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:29.127873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:29.134480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:29.135211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:29.135322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:29.162401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:29.163614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:29.237060 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:29.261013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899a27a-513c-4f57-945b-3b4bd3177233 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:29.288396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:29.379161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899a27a-513c-4f57-945b-3b4bd3177233 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:29.417510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb3021-da4a-4609-bcd8-4c8f0d336a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:29.418137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb3021-da4a-4609-bcd8-4c8f0d336a1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:29.418422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb3021-da4a-4609-bcd8-4c8f0d336a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:29.418693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb3021-da4a-4609-bcd8-4c8f0d336a1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:29.419969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb3021-da4a-4609-bcd8-4c8f0d336a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:29.421469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb3021-da4a-4609-bcd8-4c8f0d336a1d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:29.423073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:29.428964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:29.459384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:29.530707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4bc9c-64fe-4736-9ad1-ed27b2a3e01e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:29.565994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3d04a-74ac-4ad8-a77c-7a0a074350f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:29.566603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3d04a-74ac-4ad8-a77c-7a0a074350f0 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:29.566881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3d04a-74ac-4ad8-a77c-7a0a074350f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:29.567158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3d04a-74ac-4ad8-a77c-7a0a074350f0 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:29.567852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3d04a-74ac-4ad8-a77c-7a0a074350f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:29.569336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3d04a-74ac-4ad8-a77c-7a0a074350f0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:29.597590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:29 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:32.773723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff43e4-c715-4a17-b0d7-ad8aac00405c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:32.774752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff43e4-c715-4a17-b0d7-ad8aac00405c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:32.775230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff43e4-c715-4a17-b0d7-ad8aac00405c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:32.776813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:32.782412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5016ba-0b60-4e38-8d8a-1d1b8d20f3c6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:32.783271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5016ba-0b60-4e38-8d8a-1d1b8d20f3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:32.783731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5016ba-0b60-4e38-8d8a-1d1b8d20f3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:32.788180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357938-a022-48c7-b0b9-b9e2a9865ff4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:32.788712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357938-a022-48c7-b0b9-b9e2a9865ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:32.789233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51357938-a022-48c7-b0b9-b9e2a9865ff4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:32.790830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 34/63] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:32.796058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ce8b8-6a75-4de6-a1a7-1a608c29ef08 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:32.796551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ce8b8-6a75-4de6-a1a7-1a608c29ef08 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:32.797067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ce8b8-6a75-4de6-a1a7-1a608c29ef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:32.822273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0d5016ba-0b60-4e38-8d8a-1d1b8d20f3c6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:32.823963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d5016ba-0b60-4e38-8d8a-1d1b8d20f3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:32.827857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-289ce8b8-6a75-4de6-a1a7-1a608c29ef08 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:32.831437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-289ce8b8-6a75-4de6-a1a7-1a608c29ef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:32.912407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 30/64] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:32.918185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c823-2bb8-4f01-b553-4f55af551f10 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:32.918471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c823-2bb8-4f01-b553-4f55af551f10 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:32.918746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c823-2bb8-4f01-b553-4f55af551f10 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:32.925801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 35/65] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:32.927125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:32.927411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:32.927688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:32.930607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 31/66] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:32 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:32.937552 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c823-2bb8-4f01-b553-4f55af551f10 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:32.954048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:32.955605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:32.970988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:32.975646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.040987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 36/67] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:33.041857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.066456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c823-2bb8-4f01-b553-4f55af551f10 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.104032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.104720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.105057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.105318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.116805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:33.122440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:33.123404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:33.124162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:33.126210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 32/68] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:33.127305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.179272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929dcdc-ee39-4005-b51a-5c3fe375496c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.224218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.236628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.238259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.239115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.239527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.247834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530fbd5-7ab7-4850-9bdb-2b8f4f241807 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.252029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:33.257408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:33.258245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:33.258994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 37/69] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:33.259612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.275138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.288499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d3115-ac40-44f9-9784-1d20be2c25f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.289218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d3115-ac40-44f9-9784-1d20be2c25f7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.289500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d3115-ac40-44f9-9784-1d20be2c25f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.289772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d3115-ac40-44f9-9784-1d20be2c25f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.300355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d3115-ac40-44f9-9784-1d20be2c25f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c35d3115-ac40-44f9-9784-1d20be2c25f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:33.305330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:33.306201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:33.306974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 33/70] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:33.309032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0c7-8309-4947-8230-91faf9cd0b50 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.309539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0c7-8309-4947-8230-91faf9cd0b50 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.309805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae0c7-8309-4947-8230-91faf9cd0b50 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.311074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 34/71] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:33.311773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0c7-8309-4947-8230-91faf9cd0b50 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.358961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03526e3a-1456-4c43-8885-48810b1066b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.391589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.392345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.392626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.392930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.393657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:33.393956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:33.395346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e176-2542-4969-908a-9bf990206926 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:33.406999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 38/72] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:33.408197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ecafe-23d7-407c-80e2-09a650958af5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.408478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ecafe-23d7-407c-80e2-09a650958af5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.408750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ecafe-23d7-407c-80e2-09a650958af5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.410202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 39/73] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:33.410681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ecafe-23d7-407c-80e2-09a650958af5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.416525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae0c7-8309-4947-8230-91faf9cd0b50 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.450172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.450789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.451068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.451333 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.452057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:33.452361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:33.453795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ee2e0-767a-482b-831f-0e1ec2c19ad7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:33.456035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.462742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 35/74] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:33.464030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.464313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.464581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.482212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:33.483426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:33.486095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.513370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ecafe-23d7-407c-80e2-09a650958af5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.547375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.548002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.548340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.548614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.549359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:33.549666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:33.550782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 36/75] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:33.551065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8cf6-b992-4f6c-b6cb-4409e1cfd7d6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:33.551563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.576880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 40/76] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:33.576880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.576880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.576880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.591890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:33.593128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:33.648567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e172-f336-4286-a5ac-03b4b1a4b6c9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.657375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 41/77] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:33.658247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.680574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.689555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.690208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.690491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.690767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.705917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:33.707386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:33.709233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 37/78] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:33.710003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.730140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.769131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca68dc-fe79-4f90-bf13-41b1bc2895a3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.802073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.802678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.802910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.803174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.804309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:33.805792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:33.807390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 42/79] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:33.808222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.815573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71740a0-9a19-4470-ba6d-7247e60d313f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.849699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.850340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.850615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.850881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.861067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:33.865988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:33.866935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:33.867680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 38/80] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:33.872611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.905612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb82e0-25bb-451c-ba81-4f53ebcce6c5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:33.921685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.938928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:33.939590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:33.939861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:33.940127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:33.941329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:33.942777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:33.944358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 43/81] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:33.945148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:33.950819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:33.977383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf53cc7-0668-4277-99e3-cd946ae8ca0f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.011281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.012142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.012271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.012454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.013663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.015122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.016707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 39/82] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:34.021719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.049194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba147ec-854d-4af0-b29c-ba81ec53b1a4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.083175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.083782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.084061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.084332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.115436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145763b-33db-4abc-b70e-38687d96a301 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.128498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.150350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.150962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.151224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.151487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.153570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.155086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.156972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.218141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.219640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.256820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 44/83] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:34.257892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.318457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 40/84] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:34.323103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.335469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.360841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de64d2-d006-4578-8ea3-57aec384af5a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.362315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.394791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.395389 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.395659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.396658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:34.396983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.398356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.406650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 45/85] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:34.407558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.422859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e851c-faef-45d1-af70-2544a7c17a37 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.456699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.457340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.457616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.457886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.504178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e35c12-80b9-4011-b99f-78cd87a6e8a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.522326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.523786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.536966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.537491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.537597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.537857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.538125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.538797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.540243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.568144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 46/86] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:34 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:34.569683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.596013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.648869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 41/87] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:34.648869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.703759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85c15e-2ba3-406f-a650-b9fe32e12769 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.747318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.748586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.749040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.749314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.750817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d434527-19a3-4a0b-ac43-da1652e227d9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.750923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.752282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.754221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 47/88] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:34.755346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.766753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.793077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.794531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.794824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.795104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.796374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.797916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.800141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 42/89] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:34.800986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.867395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:34.879669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0edd-2936-4e26-bd3a-050deaa73cd0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.926025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.926685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.926960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.927228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.929091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.930808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.933398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 48/90] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:34.934166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:34.935523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a8ce-1128-4b57-b6ba-360cd4104ef2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:34.975834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:34.976983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:34.977534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:34.977990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:34.979447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:34.981486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:34.983545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 43/91] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:34.984860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:35.040689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:35.077554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ed769-c40b-47d3-a306-da35b864c706 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:35.095631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61f509-a94e-4c34-9806-c1d69cb2dadf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:35.122850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:35.123398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:35.123669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:35.124176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:35.125384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=1e97c57f35af41ef8cc68743b7a473c5) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:35.125699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:35.127501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:35.133975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:35.134842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:35.135305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:35.135809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:35.136455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 49/92] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/role_assignments?user.id=1e97c57f35af41ef8cc68743b7a473c5&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:35.137136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=a62909c025a448a2a69b126becb412ba) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:35.137308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:35.137906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:35.139611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:35.146273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 44/93] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:34 2022] GET /identity/v3/role_assignments?user.id=a62909c025a448a2a69b126becb412ba&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:35.147627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:35.166856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:35.266677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc3765-4fad-4c16-95ca-96fe5df986b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:35.295227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8b5f3-e3fe-444b-865d-f6fa319ed322 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:35.304679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:35.305510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:35.305884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:35.306254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:35.313753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:35.337605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:35.338500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:35.338922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:35.339334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:35.340438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] RBAC: Adding query filter params (role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=1e97c57f35af41ef8cc68743b7a473c5) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:35.341043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:35.342702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:35.349442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 45/94] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:35 2022] GET /identity/v3/role_assignments?user.id=1e97c57f35af41ef8cc68743b7a473c5&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c31497969e7944d1ae71b619ace9c773 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:35.389417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:35.390951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:35.415753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:35.508552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 50/95] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:35.514561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:35.630716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3b8b9-c89a-4974-8d68-584dd4c81b8e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:35.665931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481fd45-606b-4cf3-b44e-edf87320f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:35.666493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fd45-606b-4cf3-b44e-edf87320f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:35.666768 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fd45-606b-4cf3-b44e-edf87320f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:35.667035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fd45-606b-4cf3-b44e-edf87320f5e9 None admin] PATH_INFO: `/v3/projects/b91fcbc3032245c493ee529509456c77` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:35.668228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481fd45-606b-4cf3-b44e-edf87320f5e9 None admin] RBAC: Authorizing `identity:get_project(project_id=b91fcbc3032245c493ee529509456c77)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:35.669709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481fd45-606b-4cf3-b44e-edf87320f5e9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:35.671559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 46/96] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:35 2022] GET /identity/v3/projects/b91fcbc3032245c493ee529509456c77 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:35.677570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:35.778828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc88fc-19e8-4c3d-8281-81dc6d806385 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:35.799579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:35.814853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:35.815488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:35.815994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:35.816682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:35.817893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=b91fcbc3032245c493ee529509456c77, user.id=1e97c57f35af41ef8cc68743b7a473c5) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:35.818196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:35.819701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd27b-8077-4af6-bd83-0e7f7065c5e4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:35.825530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 51/97] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:35 2022] GET /identity/v3/role_assignments?user.id=1e97c57f35af41ef8cc68743b7a473c5&scope.project.id=b91fcbc3032245c493ee529509456c77&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:37.111559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93addc5-cf07-4d9b-9abd-c840c40e08d4 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:37.112114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93addc5-cf07-4d9b-9abd-c840c40e08d4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:37.112114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93addc5-cf07-4d9b-9abd-c840c40e08d4 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:37.113781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 47/98] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:37.119096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:37.119471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:37.119651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:37.142888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:37.144085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:37.235636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 52/99] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:37.269655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93addc5-cf07-4d9b-9abd-c840c40e08d4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:37.325560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:37.388005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93addc5-cf07-4d9b-9abd-c840c40e08d4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:37.435135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a116f7-2a6b-48d9-bbfa-3e4fc83cc4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:37.435787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a116f7-2a6b-48d9-bbfa-3e4fc83cc4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:37.436064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a116f7-2a6b-48d9-bbfa-3e4fc83cc4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:37.436339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a116f7-2a6b-48d9-bbfa-3e4fc83cc4d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:37.437497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a116f7-2a6b-48d9-bbfa-3e4fc83cc4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:37.439012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a116f7-2a6b-48d9-bbfa-3e4fc83cc4d3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:37.440625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 48/100] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:37.446521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:37.571183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bfe47-9a02-4450-9eaa-e2006933144f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:37.599600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:37.633806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0776b6-1000-4b66-8c40-da0b530d7aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:37.634473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0776b6-1000-4b66-8c40-da0b530d7aae None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:37.634757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0776b6-1000-4b66-8c40-da0b530d7aae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:37.635031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0776b6-1000-4b66-8c40-da0b530d7aae None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:37.635728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0776b6-1000-4b66-8c40-da0b530d7aae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:37.637241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0776b6-1000-4b66-8c40-da0b530d7aae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:37.691604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 53/101] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:23:37 2022] POST /identity/v3/projects => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:37.959495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa319b9-0aac-42cf-acc7-5eb52fb76640 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:37.959837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa319b9-0aac-42cf-acc7-5eb52fb76640 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:37.959950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa319b9-0aac-42cf-acc7-5eb52fb76640 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:37.961628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 49/102] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:37.974743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead60c1-418b-4663-af16-0474e0ccc6fa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:37.975043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead60c1-418b-4663-af16-0474e0ccc6fa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:37.975303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead60c1-418b-4663-af16-0474e0ccc6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:37.997525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8ead60c1-418b-4663-af16-0474e0ccc6fa None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:37.998724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8ead60c1-418b-4663-af16-0474e0ccc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:38.082586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 54/103] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:38.115838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa319b9-0aac-42cf-acc7-5eb52fb76640 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.186965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0aa75-903e-41f4-9148-f7a15017210e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.187499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0aa75-903e-41f4-9148-f7a15017210e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.187499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0aa75-903e-41f4-9148-f7a15017210e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.189001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 55/104] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:38.194590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.194877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.195156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.228870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:38.228870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:38.279065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa319b9-0aac-42cf-acc7-5eb52fb76640 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:38.308555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:38.353306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:38.353965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.354244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.354518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.368431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 56/105] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:38.377385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:38.383037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:38.383843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:38.384590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 50/106] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:38.390178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.404472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.450664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:38.557630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce70cd-64e8-4a17-9e78-0e008ce39f56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:38.568391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e53bf-d260-4df7-8df9-7466e3abf886 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:38.595567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636966c-d885-43c9-8acf-1040354699ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:38.596399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636966c-d885-43c9-8acf-1040354699ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.596780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636966c-d885-43c9-8acf-1040354699ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.597180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636966c-d885-43c9-8acf-1040354699ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.603626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:38.604412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.604891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.605300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.606158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:38.606597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:38.608127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65dc4-976c-45b2-9188-1e76a5f9a26f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:38.608946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636966c-d885-43c9-8acf-1040354699ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6636966c-d885-43c9-8acf-1040354699ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:38.614589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:38.615449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:38.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:38.616817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 51/107] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:38.617729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672590eb-43a8-4edc-aef5-4c2cc54f6ef2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.618012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672590eb-43a8-4edc-aef5-4c2cc54f6ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.618281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672590eb-43a8-4edc-aef5-4c2cc54f6ef2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.620554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 57/109] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:38.622453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a7823-91e1-4274-b56f-a5665e7d14e0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.622942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a7823-91e1-4274-b56f-a5665e7d14e0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.623427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a7823-91e1-4274-b56f-a5665e7d14e0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.623795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 52/109] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:38.624801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672590eb-43a8-4edc-aef5-4c2cc54f6ef2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.625690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 58/110] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:38.630906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.631957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.632453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.650833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:38.652343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:38.664406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:38.728434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 59/111] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:38.729632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.751540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:38.758550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672590eb-43a8-4edc-aef5-4c2cc54f6ef2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:38.800894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:38.801515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.801791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.802063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.802783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:38.803082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:38.804695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8beaa9-fe8c-44b5-808e-539b9b49150a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:38.813424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 53/112] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:38.814520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c855-e3ed-462c-b9d9-dc422ab723e7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.814801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c855-e3ed-462c-b9d9-dc422ab723e7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.815072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c855-e3ed-462c-b9d9-dc422ab723e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.833947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b02c855-e3ed-462c-b9d9-dc422ab723e7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:38.835187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b02c855-e3ed-462c-b9d9-dc422ab723e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:38.839846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a3233-b56f-4f0a-8ced-ce235afc7468 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:38.876728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:38.877409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.877689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.877960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] PATH_INFO: `/v3/users/bd7d09194a2a4799aa59dc2a5f760a19` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.892539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] RBAC: Authorizing `identity:get_user(user_id=bd7d09194a2a4799aa59dc2a5f760a19)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:38.894050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:38.895736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 60/113] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/users/bd7d09194a2a4799aa59dc2a5f760a19 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:38.900935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.917222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 54/114] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:38.917222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7461e93-2694-4751-bee0-4f56153e857e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:38.917222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7461e93-2694-4751-bee0-4f56153e857e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:38.917222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7461e93-2694-4751-bee0-4f56153e857e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:38.917222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 55/115] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:38.917222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7461e93-2694-4751-bee0-4f56153e857e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:38.938362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:38.991130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.016317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee2fc7-37fe-46fc-9dca-7b74bb48ed6f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.016852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7461e93-2694-4751-bee0-4f56153e857e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.052734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.053404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.053683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.053960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.055129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.055617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d609d0-876a-4a47-af08-c67ab2b25216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.056300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d609d0-876a-4a47-af08-c67ab2b25216 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.056572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.056673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d609d0-876a-4a47-af08-c67ab2b25216 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.056906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d609d0-876a-4a47-af08-c67ab2b25216 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.058412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 56/116] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:39.059008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.074343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d609d0-876a-4a47-af08-c67ab2b25216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c9d609d0-876a-4a47-af08-c67ab2b25216 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:39.079277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:39.080334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:39.081632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 61/117] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:23:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:39.086699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.086972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.087225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.104554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:39.105763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:39.164375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.172729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d965e5-246c-4c10-b1cc-ce5da3845881 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.175725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 62/118] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:39.182095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.207566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.208214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.208486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.208846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.269254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.281185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.282708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.310998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198e5b1-f8cb-4fca-a90d-2963dbe2262e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.346695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.347272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.347556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.347830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.360430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:39.366584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:39.367401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:39.368145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:39.369224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 63/119] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:39.369565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.388667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.397802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 57/120] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:39.403132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.471048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d70711-89b6-42b0-bd69-d5203c65997d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.496077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.507104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.507806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.508086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.508357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.508793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c229e-d03e-4363-a21c-b7428e45c71c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.509603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.511060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.512792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 64/121] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:39.513548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.545199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.545898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.546175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.546442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.547152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:39.547449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.548949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.556993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 58/122] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:39.562445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.621422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f224e3-97b4-4299-bce3-3776a3595398 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.626902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.657318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.658032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.658314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.658590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.669166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:39.674750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc242d6b-6d62-4481-9cc3-aad2b2a41045 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:39.675410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:39.676246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:39.677061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 65/123] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:39.679215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.711613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.712232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.712607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.712778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.713513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:39.713811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.715200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.724511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 59/124] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:39.725329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.740103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.790079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d31f28-cfc1-450e-89c7-976a2ea51c01 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.828410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.829097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.829409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.829678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.830855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.832333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.834046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 66/125] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:39.838983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.842007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cbecb-0a80-4406-a53b-162e2e3cc0d6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:39.860246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:39.881813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:39.883162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:39.883581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:39.884011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/a62909c025a448a2a69b126becb412ba/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:39.886807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2adfb93d804882ae83c5e3723892e1, user_id=a62909c025a448a2a69b126becb412ba, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:39.888409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:39.904637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 60/126] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:39 2022] PUT /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/a62909c025a448a2a69b126becb412ba/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:39.905483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:39.971710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff6516-90e3-46c4-a743-5a8aaea93970 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.006897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.023486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.024658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.024976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.025386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.027083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.029530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.031548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b61620-78f1-4f62-a17a-6546046fa592 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.036026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 67/127] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.037196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.074424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.075013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.075289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.075617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.076341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:40.076663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.078075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.087854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 61/128] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.088678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.151069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.186715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e331912-15bd-45eb-bb30-768c5e6f6252 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.222184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbd3aa-433d-4f01-9fa4-9f7d48a8e608 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.232198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.236859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.237203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.237487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.239992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.240882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.242894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 68/129] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.243777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.267391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.268124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.268410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.268684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.269598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] RBAC: Adding query filter params (role.id=b59ebdb8faee4b45ad7b9703b0ace0e4, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=bd7d09194a2a4799aa59dc2a5f760a19) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:40.269908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.271521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.278066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 62/130] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/role_assignments?user.id=bd7d09194a2a4799aa59dc2a5f760a19&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.278804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.310160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.362769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06df98-9da3-454d-85ac-ffeffe2faf84 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.401728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.401728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.401728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.402051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.403287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.404803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.406555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 69/131] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.407521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.416548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da987690-abf0-43d1-b41d-2679c6e517e6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.426983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.456455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706681-919e-4cc2-ba2c-5f47edfc81b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.457336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706681-919e-4cc2-ba2c-5f47edfc81b7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.458826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706681-919e-4cc2-ba2c-5f47edfc81b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.459218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706681-919e-4cc2-ba2c-5f47edfc81b7 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.460516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706681-919e-4cc2-ba2c-5f47edfc81b7 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.462275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706681-919e-4cc2-ba2c-5f47edfc81b7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.464312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 63/132] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.465911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fca0c3-68b7-400a-85e8-a9997ad52652 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.465911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fca0c3-68b7-400a-85e8-a9997ad52652 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.466191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fca0c3-68b7-400a-85e8-a9997ad52652 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.467416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 64/133] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:40.468079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fca0c3-68b7-400a-85e8-a9997ad52652 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.516750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ca0b9-e8ae-4565-ac5d-978f91e0cbf1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.546244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.555802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a09b01-ec6c-4dcb-9428-bdbfba4d2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.556618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09b01-ec6c-4dcb-9428-bdbfba4d2db5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.557016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09b01-ec6c-4dcb-9428-bdbfba4d2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.557410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09b01-ec6c-4dcb-9428-bdbfba4d2db5 None admin] PATH_INFO: `/v3/projects/b91fcbc3032245c493ee529509456c77` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.558679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a09b01-ec6c-4dcb-9428-bdbfba4d2db5 None admin] RBAC: Authorizing `identity:get_project(project_id=b91fcbc3032245c493ee529509456c77)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.560256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a09b01-ec6c-4dcb-9428-bdbfba4d2db5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.562269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 70/134] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/projects/b91fcbc3032245c493ee529509456c77 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.563479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.564030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.564395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.583109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:40.584506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:40.593084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fca0c3-68b7-400a-85e8-a9997ad52652 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.628381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.629260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.629657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.630017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/1e97c57f35af41ef8cc68743b7a473c5/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.632748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2adfb93d804882ae83c5e3723892e1, user_id=1e97c57f35af41ef8cc68743b7a473c5, role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.634370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.656973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 65/135] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:40 2022] PUT /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/1e97c57f35af41ef8cc68743b7a473c5/roles/c31497969e7944d1ae71b619ace9c773 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:40.657969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.687270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 71/136] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:40.688176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.727674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.863685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f72e11-ccab-4b77-b648-4f63643190aa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.875875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ae969-3620-4b69-8e5b-36e93033944c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:40.905850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.906590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.906886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.907224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:40.907761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.908997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.912843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.912843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 66/137] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:40.913151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:40.921303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:40.921990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:40.922269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:40.922543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] PATH_INFO: `/v3/projects/b91fcbc3032245c493ee529509456c77/users/1e97c57f35af41ef8cc68743b7a473c5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:40.925880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] RBAC: Authorizing `identity:create_grant(project_id=b91fcbc3032245c493ee529509456c77, user_id=1e97c57f35af41ef8cc68743b7a473c5, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:40.927738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:40.946374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 72/138] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:40 2022] PUT /identity/v3/projects/b91fcbc3032245c493ee529509456c77/users/1e97c57f35af41ef8cc68743b7a473c5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:40.947672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:41.044582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:41.054147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899276e-a0c3-48f5-905b-0911b69e26c8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:41.088001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fca45-b976-431f-b860-0ff4f28e4a82 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:41.144121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:41.144837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:41.145118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:41.145390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:41.146764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:41.149878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:41.152806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 67/139] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:41.158955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:41.176812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4b788-e7f4-4906-b63c-5bb8ba64bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:41.177585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4b788-e7f4-4906-b63c-5bb8ba64bb3f None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:41.177861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4b788-e7f4-4906-b63c-5bb8ba64bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:41.180007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4b788-e7f4-4906-b63c-5bb8ba64bb3f None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/1e97c57f35af41ef8cc68743b7a473c5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:41.185484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4b788-e7f4-4906-b63c-5bb8ba64bb3f None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2adfb93d804882ae83c5e3723892e1, user_id=1e97c57f35af41ef8cc68743b7a473c5, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:41.186963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4b788-e7f4-4906-b63c-5bb8ba64bb3f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:41.223443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 73/140] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:40 2022] PUT /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/1e97c57f35af41ef8cc68743b7a473c5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:41.334930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd219-60a8-44be-9d6b-d6b06549c25b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:41.393109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b6b94-4d78-44f6-8d35-47e39190c4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:41.394210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b6b94-4d78-44f6-8d35-47e39190c4ff None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:41.394721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b6b94-4d78-44f6-8d35-47e39190c4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:41.395196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b6b94-4d78-44f6-8d35-47e39190c4ff None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:41.396107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b6b94-4d78-44f6-8d35-47e39190c4ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:41.397836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b6b94-4d78-44f6-8d35-47e39190c4ff None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:41.436274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 68/141] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:41 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:42.592477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3bdf-18e1-4559-814c-4c1fd1b9c9e4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:42.592989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3bdf-18e1-4559-814c-4c1fd1b9c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:42.592989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e3bdf-18e1-4559-814c-4c1fd1b9c9e4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:42.594623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 74/142] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:42.600234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea46d-9541-477e-bea0-86b66cbea7d5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:42.601191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea46d-9541-477e-bea0-86b66cbea7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:42.601689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea46d-9541-477e-bea0-86b66cbea7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:42.629059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-691ea46d-9541-477e-bea0-86b66cbea7d5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:42.630686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-691ea46d-9541-477e-bea0-86b66cbea7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:42.654758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:42.713677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 69/143] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:42.738594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3bdf-18e1-4559-814c-4c1fd1b9c9e4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:42.790484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:42.847365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e3bdf-18e1-4559-814c-4c1fd1b9c9e4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:42.868596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff239de-f16d-4bb1-a681-fe66add57066 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:42.868961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff239de-f16d-4bb1-a681-fe66add57066 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:42.869202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff239de-f16d-4bb1-a681-fe66add57066 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:42.871509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 70/144] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:42.883839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:42.884639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:42.885390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:42.894341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:42.895000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:42.895276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:42.895543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:42.915102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:42.917831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:42.918321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:42.924002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:42.924951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:42.925729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 75/145] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:42.951965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.004679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 71/146] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:43.041112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.109500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47bbe3-fe8f-4e5a-b0df-1c74001ffbfe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.112345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:43.156731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.157758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.158209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.158649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.159578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:43.160047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:43.161698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:43.163553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:43.171474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 76/147] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:43.181683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db925920-061d-4d76-abcc-e6f65bf5a95d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.193871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.238140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.238803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.239083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.239354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.254998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:43.260573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:43.261473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:43.262249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:43.266292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 72/148] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:43.271853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.327423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b43840-2415-4134-bcb2-7c3394b1290b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.383288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.383976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.384328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.384681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] PATH_INFO: `/v3/users/a62909c025a448a2a69b126becb412ba` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.389503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] RBAC: Authorizing `identity:get_user(user_id=a62909c025a448a2a69b126becb412ba)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:43.391076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:43.397551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 77/149] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:43 2022] GET /identity/v3/users/a62909c025a448a2a69b126becb412ba => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:43.413869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.414668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0f5c2-af08-450a-afe1-2f51486da542 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.456510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.457397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.457799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.458220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.459086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:43.459510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:43.461090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:43.476093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 73/150] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:43.480855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:43.481909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.498053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:43.580785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4550-2c6f-4fe3-81a9-92dee7034b45 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.613720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2ad6-aa74-4ae4-bd0d-0d785805249b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.638968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bad33-b400-41aa-ab39-117fcad59a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.639630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bad33-b400-41aa-ab39-117fcad59a4e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.639981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bad33-b400-41aa-ab39-117fcad59a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.640323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bad33-b400-41aa-ab39-117fcad59a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.666354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.667394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.667851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.668284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] PATH_INFO: `/v3/users/bd7d09194a2a4799aa59dc2a5f760a19` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.671454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=bd7d09194a2a4799aa59dc2a5f760a19)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:43.673669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:43.676060 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 74/151] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:43 2022] GET /identity/v3/users/bd7d09194a2a4799aa59dc2a5f760a19 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:43.681079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.754963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bad33-b400-41aa-ab39-117fcad59a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:43.760789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bad33-b400-41aa-ab39-117fcad59a4e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:43.800795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:43.813515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:43.821254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d3fe-3495-4c58-8ab2-6c67d069b7e4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:43.881902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2f895-4181-42e7-94b9-724d9abb865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:43.882873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f895-4181-42e7-94b9-724d9abb865b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.883326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f895-4181-42e7-94b9-724d9abb865b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.883781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f895-4181-42e7-94b9-724d9abb865b None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.885416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f895-4181-42e7-94b9-724d9abb865b None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:43.887149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f895-4181-42e7-94b9-724d9abb865b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:43.889096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 75/152] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:43 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:43.890481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c17f1-f74a-4533-883d-f81f718d1308 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.890902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c17f1-f74a-4533-883d-f81f718d1308 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.891350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c17f1-f74a-4533-883d-f81f718d1308 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.892851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 76/153] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:43.898985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c17f1-f74a-4533-883d-f81f718d1308 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:43.912227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 78/154] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:43.913538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:43.913825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:43.914100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:43.949935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:43.951160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:44.067386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 79/155] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:44.068108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:44.069559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c17f1-f74a-4533-883d-f81f718d1308 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:44.122020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:44.123002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.123473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.123918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/bd7d09194a2a4799aa59dc2a5f760a19/roles/b59ebdb8faee4b45ad7b9703b0ace0e4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.130099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2adfb93d804882ae83c5e3723892e1, user_id=bd7d09194a2a4799aa59dc2a5f760a19, role_id=b59ebdb8faee4b45ad7b9703b0ace0e4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:44.132147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:44.156602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 77/156] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:43 2022] PUT /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1/users/bd7d09194a2a4799aa59dc2a5f760a19/roles/b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:23:44.157777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:44.163436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:44.185898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:44.269193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27299179-a8b8-42cd-b025-b92e423e5e4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:44.321667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eaf0d-16fa-4cdc-961c-737f5f1ceb28 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:44.369591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:44.370267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.370543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.370810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.372059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:44.373608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:44.375619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 80/157] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:44 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:44.389883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:44.418678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:44.419264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.419541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.419792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.449814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:44.455638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:44.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:44.457454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 78/158] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:23:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:44.467676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:44.616780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc487cac-4149-4acd-90cb-9377a5821b25 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:44.618506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:44.686356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:44.700703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:44.701350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.701621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.701888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.703968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=a62909c025a448a2a69b126becb412ba) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:44.704388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:44.705882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6193ba5-2a56-4746-818d-032e4a292e28 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:44.708312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c40d-5157-4a6b-9b9d-40b4b337d8a7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:44.719226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 81/159] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:44 2022] GET /identity/v3/role_assignments?user.id=a62909c025a448a2a69b126becb412ba&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:44.720506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8876ba-ad32-4e46-b26b-a377ef4d004d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.720798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8876ba-ad32-4e46-b26b-a377ef4d004d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.721073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8876ba-ad32-4e46-b26b-a377ef4d004d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.722426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 82/160] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:44.728697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.728994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.729260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.771596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:44.772819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:44.794900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:44.795543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:44.795825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:44.796099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:44.796853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:44.797158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:44.798646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:44.816529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 79/161] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:23:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:44.824483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:44.926474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 83/162] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:44.988175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:45.024220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a174ab2-b18b-4538-afcf-fe3258dfa5fc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:45.069151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:45.100178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:45.100909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.101189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.101465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.102773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:45.104244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:45.106234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 80/163] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:44 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:45.116438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:45.215325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57476940-b670-422c-83d0-16a876a8c1e9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:45.282338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:45.291545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934f2e-866b-45c4-82d0-6def0cd5c067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:45.292745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934f2e-866b-45c4-82d0-6def0cd5c067 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.293041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934f2e-866b-45c4-82d0-6def0cd5c067 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.293408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934f2e-866b-45c4-82d0-6def0cd5c067 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.317791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290f095-0844-4127-b465-bdae6fb3b434 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:45.318347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934f2e-866b-45c4-82d0-6def0cd5c067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-07934f2e-866b-45c4-82d0-6def0cd5c067 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:45.327559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:45.328426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:45.329330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:45.332834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 84/164] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:45.334237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d92d7-43a0-40a3-b309-edeef4489254 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.334588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d92d7-43a0-40a3-b309-edeef4489254 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.334868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d92d7-43a0-40a3-b309-edeef4489254 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.336359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 85/165] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:45.341045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d92d7-43a0-40a3-b309-edeef4489254 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:45.402525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:45.403242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.403520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.403784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.501509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d92d7-43a0-40a3-b309-edeef4489254 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:45.558182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:45.558352 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:45.559000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.559277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.559549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.560280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:45.560583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:45.562005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df6563-ca5a-45d3-b2c9-f4087bbe9fd6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:45.575265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 86/166] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:45.576378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd379b24-5890-435c-b938-52756865b956 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.576716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd379b24-5890-435c-b938-52756865b956 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.577025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd379b24-5890-435c-b938-52756865b956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.586748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:45.588291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:45.606266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fd379b24-5890-435c-b938-52756865b956 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:45.607454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fd379b24-5890-435c-b938-52756865b956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:45.710954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:45.720930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 87/167] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:45.721593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd379b24-5890-435c-b938-52756865b956 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:45.843148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd379b24-5890-435c-b938-52756865b956 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:45.858409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 81/168] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:45.859421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:45.882729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:45.883395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:45.883668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:45.883987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:45.885536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:45.887012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:45.888834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 88/169] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:45 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:45.889423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:45.985116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.005919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.015136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf077f85-10d6-49ae-8bf2-bacb0f3ce5db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.077763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.078411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.078678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.078939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.092413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1c753-1b99-42aa-928d-581714567950 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.104283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:46.109587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:46.110541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:46.111310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:46.114053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 82/170] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:46.116879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:46.150464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b22aa-1234-4869-a0c9-ec9c1f0bdcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.151112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b22aa-1234-4869-a0c9-ec9c1f0bdcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.151380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b22aa-1234-4869-a0c9-ec9c1f0bdcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.151646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b22aa-1234-4869-a0c9-ec9c1f0bdcf4 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.152987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b22aa-1234-4869-a0c9-ec9c1f0bdcf4 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:46.154437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b22aa-1234-4869-a0c9-ec9c1f0bdcf4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:46.156111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 89/171] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:45 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:46.162806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc090-5906-4b49-9b11-32ab2f81f78b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.163075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc090-5906-4b49-9b11-32ab2f81f78b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.163338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc090-5906-4b49-9b11-32ab2f81f78b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.164599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 90/172] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:46.167912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc090-5906-4b49-9b11-32ab2f81f78b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:46.281363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87df1a-2acf-4c6e-b719-b7874d4ea200 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.338120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9dc17-487f-464b-ab17-e1ca8dfdc765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.339711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9dc17-487f-464b-ab17-e1ca8dfdc765 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.339985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9dc17-487f-464b-ab17-e1ca8dfdc765 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.340253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9dc17-487f-464b-ab17-e1ca8dfdc765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.371973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc090-5906-4b49-9b11-32ab2f81f78b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.379444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.389945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.418297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.419142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.419735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.420091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.420917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:46.421347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:46.423227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:46.434056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 91/173] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:46.434983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:46.444125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9dc17-487f-464b-ab17-e1ca8dfdc765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:46.445898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9dc17-487f-464b-ab17-e1ca8dfdc765 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:46.545598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3148e-adea-4994-b626-f37340b18b66 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.550826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 83/174] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:46.552083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ee92-7f26-465a-b14d-4a357942fc0a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.552800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ee92-7f26-465a-b14d-4a357942fc0a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.553084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4ee92-7f26-465a-b14d-4a357942fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.571261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-92f4ee92-7f26-465a-b14d-4a357942fc0a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:46.572458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-92f4ee92-7f26-465a-b14d-4a357942fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:46.578820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.579475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.579747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.580013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.580820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] RBAC: Adding query filter params (role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=1e97c57f35af41ef8cc68743b7a473c5) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:46.581123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:46.582554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f05427-63f2-4ca3-887f-104477dfb76f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:46.597256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 92/175] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/role_assignments?user.id=1e97c57f35af41ef8cc68743b7a473c5&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c31497969e7944d1ae71b619ace9c773 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:46.597256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d20508-9a58-40b5-9a9e-5fe57cacf1ed None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.597256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d20508-9a58-40b5-9a9e-5fe57cacf1ed None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.597256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d20508-9a58-40b5-9a9e-5fe57cacf1ed None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.597256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 93/176] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:46.597256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d20508-9a58-40b5-9a9e-5fe57cacf1ed None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:46.634556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.642256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.658443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 84/177] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:46.660143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a715c-9aeb-4c73-91fd-cf45761e63f5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.660574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a715c-9aeb-4c73-91fd-cf45761e63f5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.661107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a715c-9aeb-4c73-91fd-cf45761e63f5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.662704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 85/178] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:46.663679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a715c-9aeb-4c73-91fd-cf45761e63f5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:46.735773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d20508-9a58-40b5-9a9e-5fe57cacf1ed None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.789062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f416470-9339-4ff0-88be-e842f07f3e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.789062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f416470-9339-4ff0-88be-e842f07f3e14 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.789062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f416470-9339-4ff0-88be-e842f07f3e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.789409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f416470-9339-4ff0-88be-e842f07f3e14 None admin] PATH_INFO: `/v3/users/1e97c57f35af41ef8cc68743b7a473c5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.790594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f416470-9339-4ff0-88be-e842f07f3e14 None admin] RBAC: Authorizing `identity:get_user(user_id=1e97c57f35af41ef8cc68743b7a473c5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:46.791997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f416470-9339-4ff0-88be-e842f07f3e14 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:46.793684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 94/179] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/users/1e97c57f35af41ef8cc68743b7a473c5 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:46.794832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.795090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.795310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a715c-9aeb-4c73-91fd-cf45761e63f5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:46.795417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.817669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:46.819125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:46.850351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45447d-13b5-4a93-8a55-1dc282552732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:46.851012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45447d-13b5-4a93-8a55-1dc282552732 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.851281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45447d-13b5-4a93-8a55-1dc282552732 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.851541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45447d-13b5-4a93-8a55-1dc282552732 None admin] PATH_INFO: `/v3/projects/b91fcbc3032245c493ee529509456c77` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.852874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45447d-13b5-4a93-8a55-1dc282552732 None admin] RBAC: Authorizing `identity:get_project(project_id=b91fcbc3032245c493ee529509456c77)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:46.854330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45447d-13b5-4a93-8a55-1dc282552732 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:46.856156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 86/180] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/projects/b91fcbc3032245c493ee529509456c77 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:46.857143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:46.857430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:46.857693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:46.888387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:46.889860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:46.909196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 95/181] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:46.910141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:46.945333 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.961468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:46.967087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 87/182] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:46.968131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.031788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7eafe-d37e-4a3d-84b0-fcf26cf9bed7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.068216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.069004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.069274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.069535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.075569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841500b1-b3bd-4b5e-8ec9-84830698c2a9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.080231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:47.085476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:47.086375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:47.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 96/183] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:47.087804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.115878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.116496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.116844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.117170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.168300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.186657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.188434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.189976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.190173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2f657-2a24-45dc-9520-f03d989445b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.225723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.226358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.226638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.226922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.227723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=b91fcbc3032245c493ee529509456c77, user.id=1e97c57f35af41ef8cc68743b7a473c5) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:47.228019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.229508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.235302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 97/184] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/role_assignments?user.id=1e97c57f35af41ef8cc68743b7a473c5&scope.project.id=b91fcbc3032245c493ee529509456c77&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:47.241407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.294607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 88/185] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:47.295605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.364948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea22e96-291e-4803-8bbd-86bae2acf781 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.382404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.405660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d212-09a5-40b8-938b-7130f6f9fb2d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.406136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.406798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.407077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.407420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.420148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:47.425633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:47.426499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:47.427344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 98/186] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:47.427977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.450128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78b0ae-ec88-4a23-ade6-4c6c597bee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.451207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78b0ae-ec88-4a23-ade6-4c6c597bee3a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.451489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78b0ae-ec88-4a23-ade6-4c6c597bee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.452824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78b0ae-ec88-4a23-ade6-4c6c597bee3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.453940 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78b0ae-ec88-4a23-ade6-4c6c597bee3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.454271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.455434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78b0ae-ec88-4a23-ade6-4c6c597bee3a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.461734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 89/187] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:47.461734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b05283-f3b8-4753-88c2-35f706f3c979 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.461734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b05283-f3b8-4753-88c2-35f706f3c979 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.461734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b05283-f3b8-4753-88c2-35f706f3c979 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.461734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 90/188] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:47.461734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b05283-f3b8-4753-88c2-35f706f3c979 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.536669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc104aa-3b35-4187-990b-18eea52eec36 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.561118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b05283-f3b8-4753-88c2-35f706f3c979 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.568869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.569487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.569758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.570021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.570723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:47.571019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.572401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.580336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 99/189] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:47.581131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.599142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.599780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.600058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.600331 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.601522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.602968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.604639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 91/190] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:47.605405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.622901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.667563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.683551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff8e0-7429-4d3f-8c74-f7850dc0e9ad None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.702297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905b8cc-79d0-4a32-8eef-ebe74851dd02 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.716244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.716901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.717165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.717430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.718139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:47.718431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.719845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdcb69-9f4c-48fa-a63c-ded76c9c1cc0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.727705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 100/191] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:47.729082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.729367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.729635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.739786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.740407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.740680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.740983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.741685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.743107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.746607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:47.747784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:47.764982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 92/192] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:47 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:47.765980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.840618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 101/193] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:47.841749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.861682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.901291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809ef85-87d8-48dc-9f84-f24df9c372c1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:47.931133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:47.944351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:47.945033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:47.945311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:47.945576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] PATH_INFO: `/v3/users/df2f1637199742b993b3163a747ea898` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:47.961005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] RBAC: Authorizing `identity:get_user(user_id=df2f1637199742b993b3163a747ea898)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:47.962159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:47.963858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 93/194] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/users/df2f1637199742b993b3163a747ea898 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:47.964697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:47.965368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52674208-73d4-4248-a3e4-385067b9adbc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.014120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.014744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.015010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.015273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.016065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=1e97c57f35af41ef8cc68743b7a473c5) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:48.016363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.018044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.026029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 102/195] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/role_assignments?user.id=1e97c57f35af41ef8cc68743b7a473c5&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:48.027088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.099712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ab0a-ead4-43ac-928e-1896a56ce84c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.141131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.147963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.148618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.149041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.149191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] PATH_INFO: `/v3/users/df2f1637199742b993b3163a747ea898` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.150408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] RBAC: Authorizing `identity:get_user(user_id=df2f1637199742b993b3163a747ea898)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.151869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.153586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 94/196] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:47 2022] GET /identity/v3/users/df2f1637199742b993b3163a747ea898 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:48.154586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.164487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d92515-ea44-4acb-a42c-9829a6c60966 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.203420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.204099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.204377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.204647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.229407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:48.238157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:48.239096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 103/197] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:48.239910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.240647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.298430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a12b1f-1649-4610-bdba-e4f17c5207cd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.337183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.337817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.338095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.338367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.352042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287105a-c119-48db-ad75-8ca1dcf0a4e9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.390640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.391275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.391621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.391920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.409449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.413397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.414903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.474722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.477563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.482201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.539999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 95/198] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:48.541372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.623123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 104/199] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:48.623790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.657175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.665699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530dd37-cf09-48d8-abac-19ef064e7876 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.711211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.711843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.712126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.712643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.713402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:48.713687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.715325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.726485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 96/200] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:48.728273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.737989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.741375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a2e2a-bda7-4206-83f3-1700e985587e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.778416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.780005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.780521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.781108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.782628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.784312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.786558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 105/201] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:48.787572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.862606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f7f38-3411-4e48-a74c-447af2b56041 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.898513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.899160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.899440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.899712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.900903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.902356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.904313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 97/202] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:48.904993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:48.908123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c76a5-b33f-4d08-b2fe-0a18c47f74b0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:48.934391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:48.954604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:48.955953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:48.956689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:48.957174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] PATH_INFO: `/v3/users/bd7d09194a2a4799aa59dc2a5f760a19` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:48.959136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] RBAC: Authorizing `identity:get_user(user_id=bd7d09194a2a4799aa59dc2a5f760a19)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:48.960821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:48.968031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 106/203] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/users/bd7d09194a2a4799aa59dc2a5f760a19 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:48.968031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:49.005304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:49.032825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb251e5-2971-4dbb-8476-c193a92b9316 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:49.074170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392807de-e761-4c34-82db-2a9d6b0d7c8b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:49.080797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:49.082113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:49.082588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:49.083045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:49.084395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=df2f1637199742b993b3163a747ea898) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:49.084925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:49.087155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:49.094481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 98/204] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/role_assignments?user.id=df2f1637199742b993b3163a747ea898&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:49.096028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:49.109834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:49.110642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:49.110989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:49.111326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:49.112269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] RBAC: Adding query filter params (role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=df2f1637199742b993b3163a747ea898) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:49.112614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:49.114228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:49.120961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 107/205] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:48 2022] GET /identity/v3/role_assignments?user.id=df2f1637199742b993b3163a747ea898&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c31497969e7944d1ae71b619ace9c773 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:49.214932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:49.235663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc60e2-b858-433d-b2cb-d32b7c719954 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:49.299957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:49.301022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:49.301508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:49.301973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:49.384012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:49.385977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:49.497435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:49.509531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 99/206] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:49.517862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:49.637800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:49.641341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92909720-15d5-4e99-bdbe-f0018613729b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:49.683992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b91cd0-e01c-40eb-9b73-4d06ec17c84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:49.684644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91cd0-e01c-40eb-9b73-4d06ec17c84d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:49.684943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91cd0-e01c-40eb-9b73-4d06ec17c84d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:49.685212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91cd0-e01c-40eb-9b73-4d06ec17c84d None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:49.686359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b91cd0-e01c-40eb-9b73-4d06ec17c84d None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:49.687817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b91cd0-e01c-40eb-9b73-4d06ec17c84d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:49.689636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 108/207] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:49 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:49.695604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:49.848884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54d7d2-c393-424c-837e-db1c10a298da None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:49.906837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:49.908301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:49.908824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:49.909411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:49.910424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] RBAC: Adding query filter params (role.id=b59ebdb8faee4b45ad7b9703b0ace0e4, scope.project.id=2c2adfb93d804882ae83c5e3723892e1, user.id=bd7d09194a2a4799aa59dc2a5f760a19) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:49.910939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:49.912589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a3dcb-79c9-4725-b9d3-0b9c63cee415 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:49.925513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 100/208] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:49 2022] GET /identity/v3/role_assignments?user.id=bd7d09194a2a4799aa59dc2a5f760a19&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:50.353934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543168e-d6cb-46f6-8cf0-f115665e60ea None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.354488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543168e-d6cb-46f6-8cf0-f115665e60ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.354488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543168e-d6cb-46f6-8cf0-f115665e60ea None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.356058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 109/209] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:50.361574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.361855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.362127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.382856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:50.384094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:50.445251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061969-9d3e-4b2e-b011-d9923eead707 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.445570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061969-9d3e-4b2e-b011-d9923eead707 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.445800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061969-9d3e-4b2e-b011-d9923eead707 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.447501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 110/210] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:50.454147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.454444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.454696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.467815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:50.479781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:50.481288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:50.495756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 101/211] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:50.526337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:50.563535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 111/212] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:50.585354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:50.640489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b7a07-0e06-4e9d-a2ec-9e587e0cebd1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:50.680501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:50.681523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.682342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.682804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.684334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:50.686635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:50.692974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 102/213] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:50.694756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:50.695179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:50.699435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9442f4-6aa2-4d3b-85a9-3194213f286e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:50.745543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:50.746107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.746380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.746647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.760846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:50.765822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:50.766791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:50.767589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 112/214] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:23:50.778418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:50.778581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:50.825685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a5b99-3465-4047-9389-fd417471d333 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:50.866785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:50.867770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.868242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.868706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.869637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:50.871326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:50.902637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 103/215] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:50 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:23:50.903805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd335-493f-4474-b19b-dc572ed273ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:50.948012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:50.948665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:50.948971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:50.949295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:50.950035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:50.950333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:50.951860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:50.962942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 113/216] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:50.968724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.080298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730971ff-b795-420b-8fb5-fa0cbda216fa None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.091838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:51.116902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed6a9e-6c06-4378-a0d6-e618c3a16e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:51.117556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed6a9e-6c06-4378-a0d6-e618c3a16e82 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.117834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed6a9e-6c06-4378-a0d6-e618c3a16e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.118105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed6a9e-6c06-4378-a0d6-e618c3a16e82 None admin] PATH_INFO: `/v3/users/f92b449570624783ac1b182a2551620e` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.132785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed6a9e-6c06-4378-a0d6-e618c3a16e82 None admin] RBAC: Authorizing `identity:get_user(user_id=f92b449570624783ac1b182a2551620e)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:51.134264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed6a9e-6c06-4378-a0d6-e618c3a16e82 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:51.136262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 104/217] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:50 2022] GET /identity/v3/users/f92b449570624783ac1b182a2551620e => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:51.141859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.166461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:51.203716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2becc2e-96da-4fcf-98d0-f5b8fd9d2eb3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.204056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2becc2e-96da-4fcf-98d0-f5b8fd9d2eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.204315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2becc2e-96da-4fcf-98d0-f5b8fd9d2eb3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.205750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 105/218] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:51.212156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.212427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.212702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.237836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:51.239446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:51.260564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e338b7-8e1b-44df-9844-f7db7234bb70 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.310944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c43144-b152-4846-9f56-2a9dc0f1fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:51.311586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c43144-b152-4846-9f56-2a9dc0f1fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.311858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c43144-b152-4846-9f56-2a9dc0f1fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.312125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c43144-b152-4846-9f56-2a9dc0f1fdd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.329917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 106/219] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:51.364375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.406924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c43144-b152-4846-9f56-2a9dc0f1fdd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:51.408510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c43144-b152-4846-9f56-2a9dc0f1fdd6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:51.436241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:51.479686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:51.484018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616450b-1f17-45f7-8eb6-bb9329208e40 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.526072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:51.526825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.527111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.527379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.538437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 114/220] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:51.544429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13047-38f1-4443-9458-3548a28dbdc2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.544566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:51.545214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13047-38f1-4443-9458-3548a28dbdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.545688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13047-38f1-4443-9458-3548a28dbdc2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:51.546268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:51.547120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:51.547981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 107/221] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:51.548632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.550756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 115/222] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:51.557036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.557662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.558148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.576545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:51.578114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:51.665342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 116/223] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:51.666078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbc164-ba35-4f6e-8801-9dd5d45cb86c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.666203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.702848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:51.703856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.704415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.704932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.706448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:51.708503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:51.710739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 108/224] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:51.711727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.729967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:51.750116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:51.806754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73226a4-87dd-4b19-a073-a6920ea9af2a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.838989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee81041-5f86-4b72-92cb-4c185ac2de20 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.842355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:51.842355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.842552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.842785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.843504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:51.843797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:51.845317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:51.858613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 117/225] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:51.858613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.891593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:51.892620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:51.893311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:51.894667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:51.911733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:51.919442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:51.920319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:51.921148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:51.923346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 109/226] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:51.923915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:51.966356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8a03b-e264-48b2-b306-88c74c6788a7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:51.980998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:52.008215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.008866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.009324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.009771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.010369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=f92b449570624783ac1b182a2551620e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:52.010669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.012115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.018179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 118/227] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/role_assignments?user.id=f92b449570624783ac1b182a2551620e&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:52.022854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:52.070349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:52.101339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81b7cc-48bd-4eb5-a34d-7ef7785219f7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.171799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.172382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.172635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.173070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] PATH_INFO: `/v3/users/df2f1637199742b993b3163a747ea898` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.174756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] RBAC: Authorizing `identity:get_user(user_id=df2f1637199742b993b3163a747ea898)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.177925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.181335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 110/228] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:51 2022] GET /identity/v3/users/df2f1637199742b993b3163a747ea898 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:52.186459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:52.227929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e8386-ad09-4c96-baf7-a1e642264b56 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.284480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.285504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.285977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.286467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.287507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:52.288055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.289897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.300482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 119/229] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:52.308536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:52.355208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:52.392697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db9311-1477-4a1e-b642-d4fedffc0b32 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.452228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99079ec-bf7c-4a33-bc0a-c353ff209454 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.463842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.464487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.464774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.465042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.466389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.469506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.471738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 111/230] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:52 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:52.478930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:52.508886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:52.528066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.528717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.529019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.529285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] PATH_INFO: `/v3/users/df2f1637199742b993b3163a747ea898` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.531536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] RBAC: Authorizing `identity:get_user(user_id=df2f1637199742b993b3163a747ea898)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.533120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.534737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 120/231] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:52 2022] GET /identity/v3/users/df2f1637199742b993b3163a747ea898 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:52.541394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:52.640570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0ea3-af08-4560-ab93-9db2a46f863a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.684915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.685594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.685869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.686132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db/users/df2f1637199742b993b3163a747ea898/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.691848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5cf7e744c64f475c8459991b467447db, user_id=df2f1637199742b993b3163a747ea898, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.694716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.716930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd7680-3c30-4350-84be-e39b885460e7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.725353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 112/232] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:52 2022] PUT /identity/v3/projects/5cf7e744c64f475c8459991b467447db/users/df2f1637199742b993b3163a747ea898/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:52.733277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:52.766907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ab149-24a8-4f43-b0d9-edf1b606ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.767561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ab149-24a8-4f43-b0d9-edf1b606ec1f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.767831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ab149-24a8-4f43-b0d9-edf1b606ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.768095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ab149-24a8-4f43-b0d9-edf1b606ec1f None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.769284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ab149-24a8-4f43-b0d9-edf1b606ec1f None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.770858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ab149-24a8-4f43-b0d9-edf1b606ec1f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.772683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 121/233] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:52 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:52.782074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:52.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:52.910413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3e3bb-2a70-4980-ba17-c1ff6b09eda4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:52.957152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a475e-301b-45a5-929f-01921ab6e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:52.957845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a475e-301b-45a5-929f-01921ab6e00e None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:52.958122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a475e-301b-45a5-929f-01921ab6e00e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:52.958392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a475e-301b-45a5-929f-01921ab6e00e None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db/users/df2f1637199742b993b3163a747ea898/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:52.962615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a475e-301b-45a5-929f-01921ab6e00e None admin] RBAC: Authorizing `identity:create_grant(project_id=5cf7e744c64f475c8459991b467447db, user_id=df2f1637199742b993b3163a747ea898, role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:52.964302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a475e-301b-45a5-929f-01921ab6e00e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:52.995490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 113/234] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:52 2022] PUT /identity/v3/projects/5cf7e744c64f475c8459991b467447db/users/df2f1637199742b993b3163a747ea898/roles/c31497969e7944d1ae71b619ace9c773 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:53.295245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd74be0-6a0e-4c15-baca-e53c25290985 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.295541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd74be0-6a0e-4c15-baca-e53c25290985 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.295733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd74be0-6a0e-4c15-baca-e53c25290985 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.297365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 122/235] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:53.305107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.305574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.305666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.327422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:53.328605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:53.385998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c63d-be53-400b-b15b-3e4b6b01c9bf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.386311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c63d-be53-400b-b15b-3e4b6b01c9bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.386534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c63d-be53-400b-b15b-3e4b6b01c9bf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.387900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 123/236] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:53.398090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.398369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.398642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.416407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 114/237] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:53.437792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:53.439061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:53.439380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:53.552318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:53.575812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 124/238] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:53.603569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39182-02a4-4678-b633-b9efeadb4671 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:53.644533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:53.645555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:53.646206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.646475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.646744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.665291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:53.670296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:53.672568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:53.674566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 115/239] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:23:53.686699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:53.752473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ea8ac-6a6e-45be-a05d-90e9c030d812 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:53.754861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:53.788931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:53.789628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.789904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.790169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.791303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:53.792770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:53.794677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 125/240] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:53.800307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:53.862940 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c0570-9ee7-43dd-8328-75da836b568a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:53.902540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846526c-fa5e-4d0d-b6af-d8ea1cf07a6a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:53.908096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:53.921735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:53.922369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.922707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.922953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.923685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:53.923986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:53.926968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:53.940211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:53.940941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:53.941218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:53.941485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:53.942216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:53.943660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:53.944920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 116/241] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:53.951680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:53.992814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 126/242] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:53 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:23:54.074638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328dbe9-c5da-4151-87f1-b568dd4df464 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:54.117813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d590de7-ba65-4214-b5cc-766eabeaa501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:54.118386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d590de7-ba65-4214-b5cc-766eabeaa501 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.118667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d590de7-ba65-4214-b5cc-766eabeaa501 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.118941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d590de7-ba65-4214-b5cc-766eabeaa501 None admin] PATH_INFO: `/v3/users/26ead742364c4abfaef039b96c326d37` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.145629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d590de7-ba65-4214-b5cc-766eabeaa501 None admin] RBAC: Authorizing `identity:get_user(user_id=26ead742364c4abfaef039b96c326d37)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:54.147125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d590de7-ba65-4214-b5cc-766eabeaa501 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:54.148942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 117/243] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:53 2022] GET /identity/v3/users/26ead742364c4abfaef039b96c326d37 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:54.154810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:54.207952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:54.311362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4189cb-f944-4260-b1c4-58c23f707999 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:54.360912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:54.361548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.361823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.362089 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.411356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691cbe3-5e1d-4a52-ad3a-3d38f49ed486 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.412022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691cbe3-5e1d-4a52-ad3a-3d38f49ed486 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.412022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691cbe3-5e1d-4a52-ad3a-3d38f49ed486 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.413489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 118/244] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:54.419726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.420005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.420269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.433859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:54.435488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:54.449266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:54.450645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:54.464472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:54.527148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:54.542467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 127/245] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:54.548192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:54.555071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 119/246] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:54.576439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:54.652439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd0a8e-d785-4083-a994-d872b6f04ccc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:54.687486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:54.701689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:54.702456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.702799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.703134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.704418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:54.705987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:54.711884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 128/247] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:54 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:54.715952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ace19a-531b-4e1b-a9ee-71bcbcce41e1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:54.719339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:54.768472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:54.769521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.770030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.770507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.782598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:54.787872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:54.788744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:54.789571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:54.792216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 120/248] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:23:54.798166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:54.836380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cfb74-cd10-45e1-af2e-a0384761144e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:54.845260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:54.872834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:54.873522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.873794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.874062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.875228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] RBAC: Adding query filter params (role.id=b59ebdb8faee4b45ad7b9703b0ace0e4, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=26ead742364c4abfaef039b96c326d37) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:54.875228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:54.876802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:54.884061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 129/249] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:54 2022] GET /identity/v3/role_assignments?user.id=26ead742364c4abfaef039b96c326d37&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:54.909271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57390f0-899c-4256-9402-cd473acd2658 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:54.947994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:54.949041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:54.949514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:54.949977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:54.950925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:54.951455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:54.953216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:54.963425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 121/250] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:54.969651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:55.037263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:55.094889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b874e-f70b-4e1c-a177-7de6e1fccd0c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:55.130960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:55.132342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.132857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.133323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] PATH_INFO: `/v3/users/f92b449570624783ac1b182a2551620e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.134694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] RBAC: Authorizing `identity:get_user(user_id=f92b449570624783ac1b182a2551620e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:55.136397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:55.138536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 130/251] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:54 2022] GET /identity/v3/users/f92b449570624783ac1b182a2551620e => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:55.144072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:55.288933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e515ffd-949f-48d2-87a1-b36328a4401c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:55.293814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:55.335852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a626-3c0a-4051-8be7-476aa48e9c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:55.336503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a626-3c0a-4051-8be7-476aa48e9c6d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.336787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a626-3c0a-4051-8be7-476aa48e9c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.337069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a626-3c0a-4051-8be7-476aa48e9c6d None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.338242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6a626-3c0a-4051-8be7-476aa48e9c6d None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:55.339690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6a626-3c0a-4051-8be7-476aa48e9c6d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:55.341702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 122/252] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:55 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:55.349000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:55.401282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4699a50-cd97-4687-9d58-9994d77dec6a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.401621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4699a50-cd97-4687-9d58-9994d77dec6a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.401823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4699a50-cd97-4687-9d58-9994d77dec6a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.403439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 123/253] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:23:55.408432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634ad5-5f7b-4f14-adaf-f8ba09d7afe0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.408743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634ad5-5f7b-4f14-adaf-f8ba09d7afe0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.409156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634ad5-5f7b-4f14-adaf-f8ba09d7afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.466187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-89634ad5-5f7b-4f14-adaf-f8ba09d7afe0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:55.466807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-89634ad5-5f7b-4f14-adaf-f8ba09d7afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:55.503838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7608c-cb6a-4fd4-922f-68e98f7f5327 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:55.532369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:55.541855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:55.542538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.542856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.543133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db/users/f92b449570624783ac1b182a2551620e/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.545144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] RBAC: Authorizing `identity:create_grant(project_id=5cf7e744c64f475c8459991b467447db, user_id=f92b449570624783ac1b182a2551620e, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:55.546618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:55.573004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 131/254] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:55 2022] PUT /identity/v3/projects/5cf7e744c64f475c8459991b467447db/users/f92b449570624783ac1b182a2551620e/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:55.615325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 124/255] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 31 18:23:55.650297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:55.682148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afede61c-4861-4d21-a49c-4eedce5d9c94 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.682799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afede61c-4861-4d21-a49c-4eedce5d9c94 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.683262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afede61c-4861-4d21-a49c-4eedce5d9c94 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.685222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 125/256] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:55.691708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.692261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.692719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.725131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:55.726959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:55.816970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339f601-d770-4e2c-8f23-bba33211b54f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:55.858069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:55.859322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:55.859597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:55.859866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:55.865100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 126/257] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:55.874495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:55.880558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:55.881789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:23:55.882573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:55.883675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 132/258] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:55.889340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:55.910274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:55.944404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:55.995256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:56.070765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c886740-a658-4d5d-8692-ff35be68a59d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.089125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd32c6-f49c-412d-b112-b7807d004aae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.132714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.133363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.133640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.133910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:56.134631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:56.134929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:56.136307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:56.148569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.149259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.149535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.149804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:56.154745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 127/259] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:56.160315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:56.174795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:56.180451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:56.181359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:23:56.182222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:56.183363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 133/260] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:23:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:23:56.190098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:56.336866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:56.344600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf704e-17ab-484c-8330-51339f61e709 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.372830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d797bd-cb3b-42ce-b7e1-5ed723f5f293 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.387250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:56.407156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.407853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.408132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.408398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] PATH_INFO: `/v3/users/df2f1637199742b993b3163a747ea898` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:56.410818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] RBAC: Authorizing `identity:get_user(user_id=df2f1637199742b993b3163a747ea898)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:56.415099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:56.418515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 128/261] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:56 2022] GET /identity/v3/users/df2f1637199742b993b3163a747ea898 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:56.423507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:56.439189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.439832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.440110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.440381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:56.441141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:56.441441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:56.443918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:56.458330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 134/262] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:23:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:56.468785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:56.606702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c1624-6c46-46f2-b668-7b8a01ee8162 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.645351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9ec03-7d89-46dd-8ed8-4a1094266399 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.666291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.666915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.667192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.667457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:56.706499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.707162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.707438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.707708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] PATH_INFO: `/v3/users/df2f1637199742b993b3163a747ea898` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:56.708916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] RBAC: Authorizing `identity:get_user(user_id=df2f1637199742b993b3163a747ea898)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:56.711270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:56.713231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 135/263] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:56 2022] GET /identity/v3/users/df2f1637199742b993b3163a747ea898 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:56.718779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:56.736009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:56.783855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:56.805889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:56.807703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:56.865983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e62694-91fd-42b8-aed9-303b44776091 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:56.931769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:56.932403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:56.932681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:56.932976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.008429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 129/264] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 18:23:57.014228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:57.035245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:57.036743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:57.075592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:57.140794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:57.175106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 136/265] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 18:23:57.176264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:57.225839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c06b6-8ea7-460a-ae0b-1a142a576dbf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:57.293506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:57.294183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.294462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.294739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.296040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:57.297519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:57.299628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 130/266] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:57 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:57.305313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:57.346536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1072-6ef0-4e99-b9ab-6fbfcfd26593 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:57.417914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:57.418493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.418776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.421415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.425433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:57.429225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:57.431008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 137/267] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:57 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:57.446804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:57.461075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:57.519182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba737c67-cc9a-4d48-9b17-a3178dd2fc30 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:57.549413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:57.556451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:57.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.557374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.557895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.558886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=df2f1637199742b993b3163a747ea898) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:57.559366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:57.560846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a2f17-de76-41c9-9844-7d668d894baf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:57.567478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 131/268] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:57 2022] GET /identity/v3/role_assignments?user.id=df2f1637199742b993b3163a747ea898&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:57.568806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89572c93-73ca-43f3-a155-616193fb47e5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.569040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89572c93-73ca-43f3-a155-616193fb47e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.569311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89572c93-73ca-43f3-a155-616193fb47e5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.570751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 132/269] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:57.591025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1a41f-f425-42a4-9aec-6f717f598fae None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.596792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1a41f-f425-42a4-9aec-6f717f598fae None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.596792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1a41f-f425-42a4-9aec-6f717f598fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.630624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a5b1a41f-f425-42a4-9aec-6f717f598fae None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:57.632484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5b1a41f-f425-42a4-9aec-6f717f598fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:57.685963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f98e51-f943-4df9-a3ad-cfaf2a1e104e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:57.758108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:57.758748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.759031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.759290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.760091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] RBAC: Adding query filter params (role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=df2f1637199742b993b3163a747ea898) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:57.760395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:57.762735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c2e2-77a1-4f35-966a-268192b69206 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:57.772115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 133/270] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:23:57.773457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760872-c681-48d8-a332-4a336d855095 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.773740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760872-c681-48d8-a332-4a336d855095 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.774013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760872-c681-48d8-a332-4a336d855095 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.775220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 138/271] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:23:57 2022] GET /identity/v3/role_assignments?user.id=df2f1637199742b993b3163a747ea898&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c31497969e7944d1ae71b619ace9c773 => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:57.781264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 134/272] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:57.787340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd37c4-88d8-4eaa-b708-2957ba7a1308 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.788005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd37c4-88d8-4eaa-b708-2957ba7a1308 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.788485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd37c4-88d8-4eaa-b708-2957ba7a1308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.820093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760872-c681-48d8-a332-4a336d855095 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:57.825698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3bbd37c4-88d8-4eaa-b708-2957ba7a1308 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:57.827137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3bbd37c4-88d8-4eaa-b708-2957ba7a1308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:57.909141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:57.914948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 139/273] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:57.926289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec6862-5c9b-4d0d-a8bb-a66137a11974 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.926559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec6862-5c9b-4d0d-a8bb-a66137a11974 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.926833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec6862-5c9b-4d0d-a8bb-a66137a11974 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.928331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 140/274] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:57.934634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:57.934919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:57.935171 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:57.966478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:57.967606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:57.975976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760872-c681-48d8-a332-4a336d855095 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.012400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.027196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.027845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.028156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.028389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.029563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.031009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.032672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 135/275] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:58.033527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.048892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 141/276] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:58.049863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.158930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011ad5-effa-4f72-bfe9-1bab37285a8c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.169103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4567846-b30e-4a70-a3ae-7dc5bd8e49d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.193779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.194414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.194682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.194943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.202025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.206580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:58.208733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.209381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.209655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.209919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.210595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:58.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:58.212411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:23:58.213612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:58.214528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.214596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 136/277] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:58.214596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.242622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 142/278] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:23:58 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:23:58.243899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.293031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.348104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb3c3d-c732-43b3-bb75-54134d16aec8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.389101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.390261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.390766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.391230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.399068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff552604-a92b-4ddb-8bc6-477ff5fcb96d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.405203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:58.411275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:58.412223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:58.413037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:23:58.415681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 137/279] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:58.423594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.439049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.439692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.439964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.440233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.442008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:58.442008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.442660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.452101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 143/280] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:58.458313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.526443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.561068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55f91-9786-4953-9311-5d5b4841a135 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.579543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.603331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27aebf-b162-4815-b0af-abb6893f4132 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.612627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.613296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.613571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.613838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.614551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:58.615052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.616211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.626533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 138/281] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:58.631895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.657776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.658739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.658739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.658990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] PATH_INFO: `/v3/users/26ead742364c4abfaef039b96c326d37` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.660866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] RBAC: Authorizing `identity:get_user(user_id=26ead742364c4abfaef039b96c326d37)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.662323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.664469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 144/282] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/users/26ead742364c4abfaef039b96c326d37 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:58.675893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.781161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f4ae-770a-4eb0-89f0-5d056fd4a3cc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.784454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1e2b-45b7-4e88-9c2f-2aa2f2e9eed8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.804868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.818355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.819001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.819292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.819538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] PATH_INFO: `/v3/users/24af309c80b449e593a51c9125b0a4f6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.826764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cb836-a8ef-40f9-8183-c15d501b38a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.827663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb836-a8ef-40f9-8183-c15d501b38a8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.828111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb836-a8ef-40f9-8183-c15d501b38a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.828547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb836-a8ef-40f9-8183-c15d501b38a8 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.830033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb836-a8ef-40f9-8183-c15d501b38a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.831641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb836-a8ef-40f9-8183-c15d501b38a8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.833843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 145/283] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:23:58.838726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] RBAC: Authorizing `identity:get_user(user_id=24af309c80b449e593a51c9125b0a4f6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.838726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.838726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 139/284] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/users/24af309c80b449e593a51c9125b0a4f6 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:58.838960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc9788-59fe-453a-8c48-810490770e60 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.838960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc9788-59fe-453a-8c48-810490770e60 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.839102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc9788-59fe-453a-8c48-810490770e60 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.841543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 146/285] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:23:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:23:58.841648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.845682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc9788-59fe-453a-8c48-810490770e60 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:58.873123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:58.944393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104c006-2f19-4350-81bb-e412fdbd3619 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.959146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc9788-59fe-453a-8c48-810490770e60 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:58.979482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ace7c8-7e4b-40b4-a5f9-ea7777ba533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:58.980166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ace7c8-7e4b-40b4-a5f9-ea7777ba533a None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:58.980443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ace7c8-7e4b-40b4-a5f9-ea7777ba533a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:58.980706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ace7c8-7e4b-40b4-a5f9-ea7777ba533a None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db/users/26ead742364c4abfaef039b96c326d37/roles/b59ebdb8faee4b45ad7b9703b0ace0e4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:58.982932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ace7c8-7e4b-40b4-a5f9-ea7777ba533a None admin] RBAC: Authorizing `identity:create_grant(project_id=5cf7e744c64f475c8459991b467447db, user_id=26ead742364c4abfaef039b96c326d37, role_id=b59ebdb8faee4b45ad7b9703b0ace0e4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:58.984122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ace7c8-7e4b-40b4-a5f9-ea7777ba533a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:58.998477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:59.003946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 140/286] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:23:58 2022] PUT /identity/v3/projects/5cf7e744c64f475c8459991b467447db/users/26ead742364c4abfaef039b96c326d37/roles/b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:23:59.003946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:59.003946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:59.003946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:59.005228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:59.005770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:59.006576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:59.007573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, user.id=24af309c80b449e593a51c9125b0a4f6, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:59.008100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:59.014147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:59.020533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 147/287] 10.222.0.49 () {56 vars in 1308 bytes} [Sun Jul 31 18:23:58 2022] GET /identity/v3/role_assignments?user.id=24af309c80b449e593a51c9125b0a4f6&scope.system=all&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:59.024819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:23:59.024819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:23:59.101474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 141/288] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:23:59.124682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:59.188011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:59.266737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8966e-382b-4fda-83c3-b777f4b8efb2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:59.311900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:59.312570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:59.312862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:59.313177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:59.325088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:23:59.330770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:23:59.331703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:23:59.332459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 148/289] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:23:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:23:59.338758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:59.366641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:59.443719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0686-8cff-4e16-8182-c2866b84a6ec None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:59.477545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:59.478169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:59.478444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:59.478719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:59.479430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:23:59.479724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:59.481116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:59.489948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 142/290] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:23:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:59.495659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:59.616008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b661d3f-ab9b-4401-a1e8-e4e09b0f497c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:59.625551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:59.652103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:59.652749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:59.653048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:59.653370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] PATH_INFO: `/v3/users/f92b449570624783ac1b182a2551620e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:59.654596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] RBAC: Authorizing `identity:get_user(user_id=f92b449570624783ac1b182a2551620e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:59.656072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:59.658125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 149/291] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:23:59 2022] GET /identity/v3/users/f92b449570624783ac1b182a2551620e => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:23:59.663821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:23:59.752856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:59.776433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4abb38-a207-4acc-a1d6-f6f4b678486e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:23:59.812346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:23:59.812996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:23:59.813318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:23:59.813585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:23:59.881925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:23:59.883244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:23:59.973575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:23:59.989099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 143/292] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:23:59.991318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:00.113294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3b9b2-5db1-458a-8654-abdf04cfe608 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:00.146395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721f8e-0d8b-45bb-81cf-c879a8696b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:00.147242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721f8e-0d8b-45bb-81cf-c879a8696b03 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:00.147671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721f8e-0d8b-45bb-81cf-c879a8696b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:00.148068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721f8e-0d8b-45bb-81cf-c879a8696b03 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:00.149358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721f8e-0d8b-45bb-81cf-c879a8696b03 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:00.150921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721f8e-0d8b-45bb-81cf-c879a8696b03 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:00.152773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 150/293] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:23:59 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:00.158931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:00.257766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fd188-8626-43ae-bed1-1722d58dde6b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:00.292651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:00.293297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:00.293570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:00.293836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:00.294625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=f92b449570624783ac1b182a2551620e) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:00.294922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:00.296355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbfcfe-d738-4807-a416-871cb2628b88 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:00.302807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 144/294] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:00 2022] GET /identity/v3/role_assignments?user.id=f92b449570624783ac1b182a2551620e&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:00.899617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b351b25-825c-48e3-a6db-70aa9061f9dc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:00.900043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b351b25-825c-48e3-a6db-70aa9061f9dc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:00.900145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b351b25-825c-48e3-a6db-70aa9061f9dc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:00.901874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 151/295] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:00.907646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:00.908123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:00.908320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:00.926752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:00.927935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:00.962831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14901008-6ee8-4918-9dfa-771045602be7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:00.963141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14901008-6ee8-4918-9dfa-771045602be7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:00.963368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14901008-6ee8-4918-9dfa-771045602be7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:00.965140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 152/296] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:00.974099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:00.974491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:00.974859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:00.982647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:00.996995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:00.998482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:01.010262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 145/297] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:01.032615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.045411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.088789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 153/298] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:01.105036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.148133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fa8fb-555a-46c2-96b3-393221bfad79 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.196203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.197627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.198105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.198570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.200529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:01.202810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:01.205989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 146/299] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:01.212000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.220833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ad493-0e92-42ca-959f-4899fa5055a7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.259383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.260458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.260957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.261468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.272714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:01.278115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:01.278951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:01.279726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:01.280533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.287371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 154/300] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:01.288689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.326038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.353305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcf8c2-ea25-4648-aa06-ce663e0c04a3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.399592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20793a30-0dda-4c1b-b935-6d38520ebe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.400342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793a30-0dda-4c1b-b935-6d38520ebe46 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.400735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793a30-0dda-4c1b-b935-6d38520ebe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.401112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793a30-0dda-4c1b-b935-6d38520ebe46 None admin] PATH_INFO: `/v3/groups` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.401934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793a30-0dda-4c1b-b935-6d38520ebe46 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:01.402638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7df0-39f5-4daf-bf22-12075aa05917 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.403543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793a30-0dda-4c1b-b935-6d38520ebe46 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:01.421168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 147/301] 10.222.0.49 () {60 vars in 1134 bytes} [Sun Jul 31 18:24:01 2022] POST /identity/v3/groups => generated 229 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:24:01.430415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f219-6c3e-4dd4-89d4-07d2e09baf59 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.430725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f219-6c3e-4dd4-89d4-07d2e09baf59 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.430971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f219-6c3e-4dd4-89d4-07d2e09baf59 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.432317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 148/302] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:01.438331 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f887184-4b04-4c34-87a6-1e42408962fa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.438628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f887184-4b04-4c34-87a6-1e42408962fa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.438898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f887184-4b04-4c34-87a6-1e42408962fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.452968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.453971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.454425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.454867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.455842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:01.456339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:01.458087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:01.459423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5f887184-4b04-4c34-87a6-1e42408962fa None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:01.461102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5f887184-4b04-4c34-87a6-1e42408962fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:01.469377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 155/303] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:01.475259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.549625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 149/304] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:01.563041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b89f6-6c1a-48a9-8108-85a034226061 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.563420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b89f6-6c1a-48a9-8108-85a034226061 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.563674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b89f6-6c1a-48a9-8108-85a034226061 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.565437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 150/305] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:01.571134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.571420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.571696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.590813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.591279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:01.592559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:01.599424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3d8ad-9533-422b-9039-e78dca3e564c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.628616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.637675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.639154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.639427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.639695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] PATH_INFO: `/v3/users/2cc64074d15f438487b8333bf1070e81` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.656080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] RBAC: Authorizing `identity:get_user(user_id=2cc64074d15f438487b8333bf1070e81)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:01.657676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:01.659372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 156/306] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/users/2cc64074d15f438487b8333bf1070e81 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:01.660197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.679493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 151/307] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:01.688959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.782457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3a0a-67e0-4130-9455-8d192bca0e16 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.790584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334018a-eed6-4ad9-b369-23530c25ba4a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.818812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.819631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.819902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.820147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.829000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:01.829693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:01.830176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:01.830446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:01.830785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:01.831236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] RBAC: Adding query filter params (role.id=b59ebdb8faee4b45ad7b9703b0ace0e4, user.id=2cc64074d15f438487b8333bf1070e81, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:01.831539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:01.833010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:01.835778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:01.836659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:01.837472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 157/308] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:01.838137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.853771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 152/309] 10.222.0.49 () {56 vars in 1308 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/role_assignments?user.id=2cc64074d15f438487b8333bf1070e81&scope.system=all&role.id=b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:01.853771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:01.861583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.888715 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:01.961314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea96bd-b058-4d8c-8bee-4b7377096dfb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:01.973709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436edd1-742e-49b9-9123-933fe30a1a33 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.014442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.015304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.015681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.015808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.016132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.016389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.016666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.016996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.017731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:02.018002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.019390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.030879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:02.035704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 153/310] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:02.036559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:02.036668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:02.037669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:02.038635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:02.040505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 158/311] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:02.048794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:02.144102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:02.146278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:02.157995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9863e-c15f-4398-9e6e-f2d4b4b47fa0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.177562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c8aa4-0d49-46d1-932b-7a2cf30a3217 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.196475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.197150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.197422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.197689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] PATH_INFO: `/v3/users/24af309c80b449e593a51c9125b0a4f6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.198848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] RBAC: Authorizing `identity:get_user(user_id=24af309c80b449e593a51c9125b0a4f6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.200282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.206462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 154/312] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:24:02 2022] GET /identity/v3/users/24af309c80b449e593a51c9125b0a4f6 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:02.207463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:02.230142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.231298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.232004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.232503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.233701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:02.234197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.236251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.252489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 159/313] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:02.254404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:02.320374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445fad4-218b-4a07-93a9-6b7b31bbaff5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.355617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7e5c-24d0-42f3-9e21-f3276b0a4498 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.357820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.359381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.359661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.359957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] PATH_INFO: `/v3/system/users/24af309c80b449e593a51c9125b0a4f6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.361443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=24af309c80b449e593a51c9125b0a4f6, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.362818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.380792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 155/314] 10.222.0.49 () {58 vars in 1334 bytes} [Sun Jul 31 18:24:02 2022] PUT /identity/v3/system/users/24af309c80b449e593a51c9125b0a4f6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:02.408296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.409356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.409896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.410334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] PATH_INFO: `/v3/users/26ead742364c4abfaef039b96c326d37` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.411928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] RBAC: Authorizing `identity:get_user(user_id=26ead742364c4abfaef039b96c326d37)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.413732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.416154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 160/315] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:24:02 2022] GET /identity/v3/users/26ead742364c4abfaef039b96c326d37 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:02.421619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:02.453615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:02.537216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f0f3-4efa-4b08-8412-23ba2d86c281 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.577036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.577558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.577830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.578093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.652128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.653629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.677863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:02.752512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 156/316] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 18:24:02.760397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:02.765016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:02.862072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c81afb-a095-4623-a71d-4202880b2417 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:02.902459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e895cb4-3a7d-4f7a-a421-043728971beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:02.903106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e895cb4-3a7d-4f7a-a421-043728971beb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:02.903379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e895cb4-3a7d-4f7a-a421-043728971beb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:02.903647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e895cb4-3a7d-4f7a-a421-043728971beb None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:02.904826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e895cb4-3a7d-4f7a-a421-043728971beb None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:02.906282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e895cb4-3a7d-4f7a-a421-043728971beb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:02.908035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 161/317] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:02 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:02.914520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:03.020384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0f110-8c9d-48a4-ae3e-beb6a692af9c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:03.044077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:03.055263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:03.056009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.056395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.056776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.057670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] RBAC: Adding query filter params (role.id=b59ebdb8faee4b45ad7b9703b0ace0e4, scope.project.id=5cf7e744c64f475c8459991b467447db, user.id=26ead742364c4abfaef039b96c326d37) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:03.058068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:03.059597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0692db-7ba3-4019-9e03-fef4e20ee785 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:03.066396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 157/318] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:02 2022] GET /identity/v3/role_assignments?user.id=26ead742364c4abfaef039b96c326d37&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:03.511492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb001e9-55e6-4e1d-be2b-45b64e671b00 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.512003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb001e9-55e6-4e1d-be2b-45b64e671b00 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.512003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb001e9-55e6-4e1d-be2b-45b64e671b00 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.513507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 162/319] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:03.519006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.519253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.519553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.539509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:03.540744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:03.612876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 158/320] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:03.635522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb001e9-55e6-4e1d-be2b-45b64e671b00 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:03.709541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:03.744416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb001e9-55e6-4e1d-be2b-45b64e671b00 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:03.780166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a410-3063-40bc-b52d-959ab8190f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:03.780729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a410-3063-40bc-b52d-959ab8190f89 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.781036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a410-3063-40bc-b52d-959ab8190f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.781307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a410-3063-40bc-b52d-959ab8190f89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.782484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a410-3063-40bc-b52d-959ab8190f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:03.784007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a410-3063-40bc-b52d-959ab8190f89 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:03.785903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 163/321] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:03.791630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:03.887771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3aeff-d6de-4652-9088-3dec120b1224 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.888146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3aeff-d6de-4652-9088-3dec120b1224 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.888327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3aeff-d6de-4652-9088-3dec120b1224 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.892795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 164/322] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:03.895075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.895579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.896046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.898031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd92d5-5e77-41a6-a35d-40e2b4726c68 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:03.915330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:03.917071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:03.918824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:03.933493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:03.934116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:03.934389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:03.934651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] PATH_INFO: `/v3/groups` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:03.935311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:03.936739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:03.953348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 159/323] 10.222.0.49 () {60 vars in 1134 bytes} [Sun Jul 31 18:24:03 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:03.998305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 165/324] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:04.022796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:04.147300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535495ec-4c24-44a3-8e6f-defe78cc0dd0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:04.185559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:04.189273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.189273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.189273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.199708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:04.205398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:04.206404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:04.208235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 160/325] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:04.212793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:04.299360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:04.333955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d37a0-75aa-47d9-8063-b2f888ee4511 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:04.370843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:04.371479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.371750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.372022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.372748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:04.373058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:04.374441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bd254-1b4c-48d6-b6e0-b4aca1e8f0aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:04.385577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 166/326] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:04.391403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:04.431090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b1cea-05d3-406b-b7f2-55f9b4d6c91f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.431286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b1cea-05d3-406b-b7f2-55f9b4d6c91f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.431548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b1cea-05d3-406b-b7f2-55f9b4d6c91f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.433199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 167/327] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:04.437948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.438325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.438599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.445963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:04.461084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:04.462287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:04.533380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad1842-cf8b-4b53-8b8a-cc18462d2d84 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:04.556101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 168/328] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:04.572872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:04.573613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.573894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.574160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] PATH_INFO: `/v3/users/2cc64074d15f438487b8333bf1070e81` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.575338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] RBAC: Authorizing `identity:get_user(user_id=2cc64074d15f438487b8333bf1070e81)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:04.576839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:04.578833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 161/329] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:24:04 2022] GET /identity/v3/users/2cc64074d15f438487b8333bf1070e81 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:04.579044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:04.585001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:04.662307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:04.710392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59885f-f359-4b09-92e4-1ec32db34247 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:04.712106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8340712-d207-4792-bf70-c92fe33f6dab None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:04.719173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:04.754071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:04.754811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.755181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.755453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] PATH_INFO: `/v3/system/users/2cc64074d15f438487b8333bf1070e81/roles/b59ebdb8faee4b45ad7b9703b0ace0e4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.758795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cc64074d15f438487b8333bf1070e81, role_id=b59ebdb8faee4b45ad7b9703b0ace0e4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:04.760092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:04.761250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:04.762189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.762465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.763213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.773433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 162/330] 10.222.0.49 () {58 vars in 1334 bytes} [Sun Jul 31 18:24:04 2022] PUT /identity/v3/system/users/2cc64074d15f438487b8333bf1070e81/roles/b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:24:04.776411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:04.781604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:04.782590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:04.783842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 169/331] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:04.789249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:04.911460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38fbae-1ab0-45f3-947a-d13f2a8dec07 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:04.950343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:04.950970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:04.951246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:04.951511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:04.952243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:04.952539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:04.954030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:04.963970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 163/332] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:04.969777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:05.083081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df65bf-1074-4f0c-ac4d-81f5009b1111 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:05.107778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:05.120958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ce83b-092e-407b-abbc-5a4f283ae70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:05.121654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce83b-092e-407b-abbc-5a4f283ae70d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:05.121932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce83b-092e-407b-abbc-5a4f283ae70d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:05.122197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ce83b-092e-407b-abbc-5a4f283ae70d None admin] PATH_INFO: `/v3/users/24af309c80b449e593a51c9125b0a4f6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:05.123359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ce83b-092e-407b-abbc-5a4f283ae70d None admin] RBAC: Authorizing `identity:get_user(user_id=24af309c80b449e593a51c9125b0a4f6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:05.124831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ce83b-092e-407b-abbc-5a4f283ae70d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:05.126921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 170/333] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:24:04 2022] GET /identity/v3/users/24af309c80b449e593a51c9125b0a4f6 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:05.133997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:05.169220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:05.243273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d246c-ce98-47b1-8f30-13976c2eea8c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:05.277965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:05.278580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:05.278849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:05.279114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:05.279999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] RBAC: Adding query filter params (role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, user.id=24af309c80b449e593a51c9125b0a4f6, scope.system=all) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:05.280319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:05.281869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ab42-b59d-4280-a3e9-461460d94163 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:05.294190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 164/334] 10.222.0.49 () {56 vars in 1308 bytes} [Sun Jul 31 18:24:05 2022] GET /identity/v3/role_assignments?user.id=24af309c80b449e593a51c9125b0a4f6&scope.system=all&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:06.620676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7b730-5cff-4908-9665-1730b510848e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:06.621190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7b730-5cff-4908-9665-1730b510848e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:06.621190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7b730-5cff-4908-9665-1730b510848e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:06.622749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 171/335] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:06.630429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:06.630939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:06.631015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:06.662924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:06.664177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:06.746572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 165/336] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:06.785414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7b730-5cff-4908-9665-1730b510848e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:06.910356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7b730-5cff-4908-9665-1730b510848e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:06.955105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5f9aa-127e-46bb-a26f-3f660f728110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:06.955797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f9aa-127e-46bb-a26f-3f660f728110 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:06.956068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f9aa-127e-46bb-a26f-3f660f728110 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:06.956333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5f9aa-127e-46bb-a26f-3f660f728110 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:06.969333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5f9aa-127e-46bb-a26f-3f660f728110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4bd5f9aa-127e-46bb-a26f-3f660f728110 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:06.976754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:06.977695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:06.979058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 172/337] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:06.983752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:07.010072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:07.046523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bbbaf-b8f4-49a2-9787-0d5398f03621 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.046852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bbbaf-b8f4-49a2-9787-0d5398f03621 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.047055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bbbaf-b8f4-49a2-9787-0d5398f03621 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.048627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 173/338] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:07.053745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.054025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.054294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.101790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:07.103000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:07.115916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec15a7-3352-428f-9634-d31eb37630f9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:07.183995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:07.206821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:07.207449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.207722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.207988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.208707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:07.209040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:07.210559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:07.222666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 174/339] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:07.239451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 166/340] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:07.245191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:07.257066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:07.401471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:07.418461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbdfab-754c-448e-b551-f61884b3f42a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:07.436030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a243a9d-d6f8-49d5-8198-6c10d8fe7c93 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:07.480020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289bed0-4f51-45e8-b395-0cb399521705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:07.480650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bed0-4f51-45e8-b395-0cb399521705 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.480946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bed0-4f51-45e8-b395-0cb399521705 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.481212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bed0-4f51-45e8-b395-0cb399521705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.497076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:07.497741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.498004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.498266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.519279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:07.526782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:07.527671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:07.528457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:07.533172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 167/341] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:07.539192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:07.575678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:07.607416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bed0-4f51-45e8-b395-0cb399521705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:07.608955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289bed0-4f51-45e8-b395-0cb399521705 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:07.700300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c262d-40b9-47f4-b345-2a411c975c2b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:07.760208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:07.760866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.761148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.761417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.762148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:07.762445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:07.763911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcd658-a8b2-4041-b186-09e04874ff82 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:07.776815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 175/342] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:07.778096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946ed80-fdd9-4923-9c8f-bbea25154e60 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.778375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946ed80-fdd9-4923-9c8f-bbea25154e60 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.778638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946ed80-fdd9-4923-9c8f-bbea25154e60 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.779906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 176/343] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:07.788722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 168/344] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:07.790487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:07.790765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:07.791033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:07.792598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946ed80-fdd9-4923-9c8f-bbea25154e60 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:07.812090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:07.823673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:07.827010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:07.941083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:07.957342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 169/345] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:07.958886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:07.978189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946ed80-fdd9-4923-9c8f-bbea25154e60 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.042687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.043410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.043661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.043920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.045196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:08.046871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:08.051114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 177/346] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:07 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:08.051894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.141315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4cec3-c625-4433-a2ef-bf5e67ec6636 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.187514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22a830-61f5-4960-9fa7-f816cdd7310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.188264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22a830-61f5-4960-9fa7-f816cdd7310c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.188538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22a830-61f5-4960-9fa7-f816cdd7310c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.188820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22a830-61f5-4960-9fa7-f816cdd7310c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.197115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:08.238671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffdf31-2f53-4822-a90c-56bfefb9a57e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.282121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22a830-61f5-4960-9fa7-f816cdd7310c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:08.283039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.283589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22a830-61f5-4960-9fa7-f816cdd7310c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:08.284124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.284600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.285091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.299024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:08.304698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:08.308012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:08.308843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:08.311403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 178/347] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:08.312321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.339795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:08.395622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 170/348] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:08.396845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce014a-517c-4b86-a9ad-807acdb6cb2c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.397135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce014a-517c-4b86-a9ad-807acdb6cb2c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.397685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce014a-517c-4b86-a9ad-807acdb6cb2c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.399520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 171/349] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:08.401593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a798e21-a565-4109-8c64-d7f6ad626b02 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.401858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a798e21-a565-4109-8c64-d7f6ad626b02 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.402123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a798e21-a565-4109-8c64-d7f6ad626b02 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.403580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 172/350] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:08.404287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a798e21-a565-4109-8c64-d7f6ad626b02 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.439088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04588de2-811c-4b71-a375-3d9281848934 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.448604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:08.479937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.480641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.481061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.481488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.490178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:08.492470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:08.496903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 179/351] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:08.498011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.526429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a798e21-a565-4109-8c64-d7f6ad626b02 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.569961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.570586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.570863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.571126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.571847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:08.572142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:08.573561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9efd9-4c7a-4b57-bc6d-fcd2c7eaa9ba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:08.582656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 173/352] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:08.583659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:08.585006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.585326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.585722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.606407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:08.607591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:08.620414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806ce0-e5ef-48a2-b687-4039326b7904 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.662560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2930ea6-259e-4fba-a88d-68e0853822b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.662560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2930ea6-259e-4fba-a88d-68e0853822b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.662560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2930ea6-259e-4fba-a88d-68e0853822b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.662560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2930ea6-259e-4fba-a88d-68e0853822b8 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.663013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2930ea6-259e-4fba-a88d-68e0853822b8 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:08.665423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2930ea6-259e-4fba-a88d-68e0853822b8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:08.667133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 180/353] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:08.668226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc654-f6f1-4583-973d-19183eaeb51e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.668589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc654-f6f1-4583-973d-19183eaeb51e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.668794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc654-f6f1-4583-973d-19183eaeb51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.681838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 174/354] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:08.682529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.693073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-075bc654-f6f1-4583-973d-19183eaeb51e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:08.694798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-075bc654-f6f1-4583-973d-19183eaeb51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:08.724273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:08.771625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 181/355] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:08.773132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a1b84-76d2-4987-90cc-4e5cb6e2db38 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.773421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a1b84-76d2-4987-90cc-4e5cb6e2db38 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.773693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a1b84-76d2-4987-90cc-4e5cb6e2db38 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.774949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 182/356] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:08.775692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a1b84-76d2-4987-90cc-4e5cb6e2db38 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.806553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0b8d1-333c-4b5c-bff3-9123edd4e3b6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.849936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.850616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.850895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.851184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.851978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=2c2adfb93d804882ae83c5e3723892e1) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:08.852281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:08.853752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:08.860303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 175/357] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c31497969e7944d1ae71b619ace9c773 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:08.861168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:08.866356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:08.919145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a1b84-76d2-4987-90cc-4e5cb6e2db38 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:08.963371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad02e8f-1bb0-4679-802f-b1c0712ae0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:08.964014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad02e8f-1bb0-4679-802f-b1c0712ae0cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:08.964289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad02e8f-1bb0-4679-802f-b1c0712ae0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:08.964555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad02e8f-1bb0-4679-802f-b1c0712ae0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:08.985731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.003792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ea33-f6bb-42d0-bc1e-c34da74cc705 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.050671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.051687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.052155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.052633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.054106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.055778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:09.058554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 176/358] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:09.059562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.060980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad02e8f-1bb0-4679-802f-b1c0712ae0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.062472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad02e8f-1bb0-4679-802f-b1c0712ae0cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:09.160460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.191794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb50c6-334b-4dda-911c-c56b1c892a63 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.202871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 183/359] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:09.204989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.205475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.205934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.227514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:09.228099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:09.249807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.250957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.251427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.251932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.264290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:09.270804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:09.271692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 18:24:09.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:09.276091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 177/360] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:09.277001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.299223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.305991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 184/361] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:09.307257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.397998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb234c-4d52-4996-a53d-3c20248932d3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.423087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.426479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c625aa7-3dfb-4a6a-a854-04794ad32565 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.445945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.446598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.446873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.447141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.459462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:09.465348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:09.466219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:09.467002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 178/362] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:09.467642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.469990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.469990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.470182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.470349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.471141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=2c2adfb93d804882ae83c5e3723892e1) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:09.471452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.472994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:09.479050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 185/363] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:09.479915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.580336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.589830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ced3-c336-46f3-b684-e50605042d43 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.590032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884587f-ea83-4376-9436-9fe7228964d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.629761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.630195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.630601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.630947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.631652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.631982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.632418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.632515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:09.632896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.633149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.634563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:09.634780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.636483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:09.638604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 179/364] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:09.639619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.650082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 186/365] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:09.651250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.694134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.760446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2db4-9fef-471b-bbc6-a208c5b73db5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.770136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603359b-1477-413a-9270-0c2955e87e7e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.795930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.796944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.797428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.797882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.809113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:09.809690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.810503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.810942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.811395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.812275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:09.812739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.815350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:09.815459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:09.816283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:09.817107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 180/366] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:09.818257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.831347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.834583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 187/367] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:09.835891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:09.946543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:09.951530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aef0c-a6bf-42ee-ba4d-60c689a37176 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.960104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ed21a-fbb3-4fd6-b249-e898a5efbe5c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:09.994542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.995508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.995944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.996345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.996741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:09.997431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:09.997704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:09.997970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] PATH_INFO: `/v3/users/2cc64074d15f438487b8333bf1070e81` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:09.998268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.999145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] RBAC: Authorizing `identity:get_user(user_id=2cc64074d15f438487b8333bf1070e81)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:09.999987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.000659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.002115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 181/368] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.002646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 188/369] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:24:09 2022] GET /identity/v3/users/2cc64074d15f438487b8333bf1070e81 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.003161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.005541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.097854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:10.126510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84e3ba-ff3e-474d-befe-1d75082cfc61 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.136801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023da2b-18a9-44ac-8074-a6d0a0ef276d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.166428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.167280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.167754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.168179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.169058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:10.169659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.170782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.171407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.171567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.171664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.171937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.181269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 182/370] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.182291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.199438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:10.246153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.247680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.299657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad0768-8d68-447c-8809-d0129ccf660d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.336974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.337866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.338279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.338746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.339766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=5cf7e744c64f475c8459991b467447db) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:10.340281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.341932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.345316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:10.350351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 189/371] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:10.350740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 183/372] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.351251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.351838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.416507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:10.466118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0d1b7-62b9-4aa7-a269-a95b2b17ec6d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.504253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.505867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.506493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.506770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.507582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] RBAC: Adding query filter params (role.id=b59ebdb8faee4b45ad7b9703b0ace0e4, user.id=2cc64074d15f438487b8333bf1070e81, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:10.507883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.509389 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.523669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 190/373] 10.222.0.49 () {56 vars in 1308 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/role_assignments?user.id=2cc64074d15f438487b8333bf1070e81&scope.system=all&role.id=b59ebdb8faee4b45ad7b9703b0ace0e4 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.524620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.588074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2e1f9-8296-4a50-b302-2c76740d85eb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.648718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82982614-3d9e-467a-b29d-669228e4813d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.655295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.655981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.656265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.656535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.673182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:10.689518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.690191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.690469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.690738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.691933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.694194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.696336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 191/374] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.702311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.779510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.785259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.793150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:10.850869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ade9f-873b-4e3d-b31a-9686e0d925fa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:10.891667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:10.892323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:10.892696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:10.892989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:10.894372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:10.895878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:10.898207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 192/375] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:10.903800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:10.922604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 184/376] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:10.928464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:11.011293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:11.081157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:11.086030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba59cb0-cca6-426b-85de-b5462dbb3b14 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:11.120895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bb6b3-1acc-4eb9-aa95-09e93ed2da3e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:11.149456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14205a-bc09-4ac7-afd8-d6e3f051c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:11.150128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14205a-bc09-4ac7-afd8-d6e3f051c33e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.150400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14205a-bc09-4ac7-afd8-d6e3f051c33e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.150669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14205a-bc09-4ac7-afd8-d6e3f051c33e None admin] PATH_INFO: `/v3/projects/57e06c47f85e41698c0742550f1a32ec` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.153436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14205a-bc09-4ac7-afd8-d6e3f051c33e None admin] RBAC: Authorizing `identity:get_project(project_id=57e06c47f85e41698c0742550f1a32ec)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:11.154949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14205a-bc09-4ac7-afd8-d6e3f051c33e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:11.156992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 185/377] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/projects/57e06c47f85e41698c0742550f1a32ec => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:11.157955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f214f-7118-4816-9a57-5aa169ff3ded None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.158234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f214f-7118-4816-9a57-5aa169ff3ded None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.158503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f214f-7118-4816-9a57-5aa169ff3ded None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.159973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 186/378] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:11.165811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.166095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.166371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.179821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:11.180692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.181154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.181573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.182688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=5cf7e744c64f475c8459991b467447db) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:11.183189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:11.184811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:11.188364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:11.189425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:11.191833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 193/379] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:10 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c31497969e7944d1ae71b619ace9c773 => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:11.192935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:11.303380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acc0e2-3972-4aba-8d46-df5d71b84a46 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:11.333179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:11.343664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:11.344873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.345160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.345437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] PATH_INFO: `/v3/groups/65ba7fc74292469ca3d5d0b10e9fc0a6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.351186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] RBAC: Authorizing `identity:get_group(group_id=65ba7fc74292469ca3d5d0b10e9fc0a6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:11.352668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:11.354773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 194/380] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:11 2022] GET /identity/v3/groups/65ba7fc74292469ca3d5d0b10e9fc0a6 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:11.362629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:11.390555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 187/381] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:11.425003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:11.492551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bcb7a-da37-4b4e-b5a1-30feab40666f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:11.563264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273941fd-608f-489c-bbfe-10c79c444941 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:11.586812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:11.589188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:11.589830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.590102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.590369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.591167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] RBAC: Adding query filter params (group.id=65ba7fc74292469ca3d5d0b10e9fc0a6, role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=57e06c47f85e41698c0742550f1a32ec) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:11.593065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:11.594942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:11.607807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:11.608662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.608970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.609240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.611946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 195/382] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:11 2022] GET /identity/v3/role_assignments?group.id=65ba7fc74292469ca3d5d0b10e9fc0a6&scope.project.id=57e06c47f85e41698c0742550f1a32ec&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:11.623793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:11.629024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:11.629943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:11.630757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 188/383] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:11.636446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:11.683536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:11.759790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08047b-d1cc-40b9-a31c-7e860a57252c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:11.807639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:11.808279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:11.808568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:11.808847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:11.809671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:11.809967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:11.811355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:11.825514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 196/384] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:11.831473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:11.977693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb52d0-e044-44d0-88ba-6aa64bdf131f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:12.015629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:12.016369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.016643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.016944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.018201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:12.019676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:12.021755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 189/385] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:11 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:12.027409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:12.156141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6451de8-5bee-479a-82a7-aaca2ec3c2b2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:12.160017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:12.195179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fead183-05d9-43f8-a405-78a2ffba610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:12.195827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead183-05d9-43f8-a405-78a2ffba610c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.196099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead183-05d9-43f8-a405-78a2ffba610c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.196405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead183-05d9-43f8-a405-78a2ffba610c None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.200046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fead183-05d9-43f8-a405-78a2ffba610c None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:12.201541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fead183-05d9-43f8-a405-78a2ffba610c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:12.203628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 197/386] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:12 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:12.209251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:12.255378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:12.348005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691564f-d794-41fb-afaa-4ae3d0b3af4b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:12.385787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:12.386461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.386727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.386990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1/groups/b52c30f30f12459ca06084d251cc43b6/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.389189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2adfb93d804882ae83c5e3723892e1, group_id=b52c30f30f12459ca06084d251cc43b6, role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:12.390641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:12.392498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff3f8-fcec-43fb-8e3c-edde6cf551e9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.392804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff3f8-fcec-43fb-8e3c-edde6cf551e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.393080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff3f8-fcec-43fb-8e3c-edde6cf551e9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.396241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 198/387] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:12.402610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.402890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.403219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.413423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 190/388] 10.222.0.49 () {58 vars in 1442 bytes} [Sun Jul 31 18:24:12 2022] PUT /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1/groups/b52c30f30f12459ca06084d251cc43b6/roles/c31497969e7944d1ae71b619ace9c773 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:12.425930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:12.427279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:12.529778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 199/389] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:12.566803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:12.689527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:12.706140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32179c-ef65-469f-b3a6-5b96c88873e9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:12.742884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:12.743525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.743795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.744060 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.754921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:12.760663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:12.761571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:12.762344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 191/390] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:12.767826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:12.888959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8de0fb-c808-4d54-be85-c3d1b811c9fd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:12.891033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:12.926434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:12.927443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:12.927990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:12.928455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:12.929409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:12.929914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:12.931493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:12.940788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 200/391] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:12.946332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.095210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5263a12-c48b-48f5-ada4-70557011c7e0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:13.140284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:13.140887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.141223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.141495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.142657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:13.144105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:13.145800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 192/392] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:12 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:13.151496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.286128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dd877-ae6f-4104-99f1-ee58cfe3c8eb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:13.335290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde7a88-89d3-4da3-9914-429a0157b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:13.335943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde7a88-89d3-4da3-9914-429a0157b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.336217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde7a88-89d3-4da3-9914-429a0157b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.336482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde7a88-89d3-4da3-9914-429a0157b1f2 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.337682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde7a88-89d3-4da3-9914-429a0157b1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:13.339125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde7a88-89d3-4da3-9914-429a0157b1f2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:13.340814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 201/393] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:13 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:13.346795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.357590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc0fe-26a4-4066-902f-8197900daad2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.357866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc0fe-26a4-4066-902f-8197900daad2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.358132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc0fe-26a4-4066-902f-8197900daad2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.359478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 202/394] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:13.365324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858522a3-cb99-4a79-aba2-2a2ead33e53a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.365609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858522a3-cb99-4a79-aba2-2a2ead33e53a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.365882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858522a3-cb99-4a79-aba2-2a2ead33e53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.371965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:13.409718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-858522a3-cb99-4a79-aba2-2a2ead33e53a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:13.410762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-858522a3-cb99-4a79-aba2-2a2ead33e53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:13.467072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:13.512973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26d890-c6ef-4376-8126-f1fd22e9ddd2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:13.544035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 203/395] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:13.545341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c344b-2f6b-4b09-9cd7-59090ce18b32 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.545620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c344b-2f6b-4b09-9cd7-59090ce18b32 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.545899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c344b-2f6b-4b09-9cd7-59090ce18b32 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.547376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 204/396] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:13.552281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.552631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.552940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.560346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:13.561382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.561878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.562340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1/groups/b52c30f30f12459ca06084d251cc43b6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.564558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2adfb93d804882ae83c5e3723892e1, group_id=b52c30f30f12459ca06084d251cc43b6, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:13.566235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:13.571008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:13.572454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:13.582277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 193/397] 10.222.0.49 () {58 vars in 1442 bytes} [Sun Jul 31 18:24:13 2022] PUT /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1/groups/b52c30f30f12459ca06084d251cc43b6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:13.583562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.655739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 205/398] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:13.669231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:13.696811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeced9-b6a2-409f-93e0-a1eced08ba03 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:13.699079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.734548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:13.735658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.736146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.736643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.748782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:13.755241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:13.756566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:13.757394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:13.760391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 194/399] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:13.766248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.855708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:13.865933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fe93c-6aae-4cf6-a908-91998fc678b2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:13.901917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:13.902561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.902834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.903099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.910823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d719ed-39e5-4f8d-9b0b-e51bcd4aef21 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:13.913685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:13.918713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:13.919658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:13.920525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 206/400] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:13.926134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:13.962514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:13.963513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:13.963795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:13.964377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:13.965256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:13.965561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:13.972786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:13.972786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:13.983650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 195/401] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:13.992791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.098815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9a5c5-6c4d-4049-bdc4-6d205069ef66 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.127054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52f7bb-1c63-4400-a13c-0273bc3fddf9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.135944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.136639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.136952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.137221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.138378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.139965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.145778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 196/402] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:13 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:14.147782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.219871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:14.237571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.238227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.238510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.238783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.239522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:14.239834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.241263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.259966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 207/403] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:14.266139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.278985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce4d10-9313-4e46-9493-37000dd8d76e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.298538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:14.316997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.317649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.317921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.318186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.319336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.320981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.327518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 197/404] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:14 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:14.328730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.401557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da6baf-3261-4cf1-a48f-11dd48beb2da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.436188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08190c21-89ee-4d70-89b5-2803a329a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.436876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08190c21-89ee-4d70-89b5-2803a329a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.437157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08190c21-89ee-4d70-89b5-2803a329a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.437429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08190c21-89ee-4d70-89b5-2803a329a2b4 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.438794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08190c21-89ee-4d70-89b5-2803a329a2b4 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.440240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08190c21-89ee-4d70-89b5-2803a329a2b4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.446741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 208/405] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:14 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:14.448164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8536b-d7fd-4367-ba76-19ec92844052 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.448460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8536b-d7fd-4367-ba76-19ec92844052 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.448725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8536b-d7fd-4367-ba76-19ec92844052 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.450242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 209/406] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:14.451060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8536b-d7fd-4367-ba76-19ec92844052 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.456967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c195-cd1b-4dd4-adc9-d5439eca4642 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.496396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:14.502978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae7043-dd01-4c5f-89e6-99b2ebb848eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.503664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7043-dd01-4c5f-89e6-99b2ebb848eb None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.503944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7043-dd01-4c5f-89e6-99b2ebb848eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.504215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7043-dd01-4c5f-89e6-99b2ebb848eb None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db/groups/b52c30f30f12459ca06084d251cc43b6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.506939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae7043-dd01-4c5f-89e6-99b2ebb848eb None admin] RBAC: Authorizing `identity:create_grant(project_id=5cf7e744c64f475c8459991b467447db, group_id=b52c30f30f12459ca06084d251cc43b6, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.508446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae7043-dd01-4c5f-89e6-99b2ebb848eb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.536922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 198/407] 10.222.0.49 () {58 vars in 1442 bytes} [Sun Jul 31 18:24:14 2022] PUT /identity/v3/projects/5cf7e744c64f475c8459991b467447db/groups/b52c30f30f12459ca06084d251cc43b6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:14.538545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.538826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.544817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.562453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:14.585960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:14.587166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:14.587602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8536b-d7fd-4367-ba76-19ec92844052 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.630706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.631817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.632093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.632442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.634213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.635797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.638020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 210/408] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:14 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:14.644862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.767601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732da7-7db8-4dcd-bec9-c1c821584f80 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.786098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 199/409] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:14.807182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.807919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:14.808198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:14.808553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db/groups/b52c30f30f12459ca06084d251cc43b6/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:14.809908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:14.810842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] RBAC: Authorizing `identity:create_grant(project_id=5cf7e744c64f475c8459991b467447db, group_id=b52c30f30f12459ca06084d251cc43b6, role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:14.812479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:14.834048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 211/410] 10.222.0.49 () {58 vars in 1442 bytes} [Sun Jul 31 18:24:14 2022] PUT /identity/v3/projects/5cf7e744c64f475c8459991b467447db/groups/b52c30f30f12459ca06084d251cc43b6/roles/c31497969e7944d1ae71b619ace9c773 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:24:14.949378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431b2e2-14ac-4ae5-92e9-cb212964dfb8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:14.983887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:14.999168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50d1e8-32cb-4dd3-8888-36c2a1ec5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:14.999870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d1e8-32cb-4dd3-8888-36c2a1ec5a66 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.000145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d1e8-32cb-4dd3-8888-36c2a1ec5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.000485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d1e8-32cb-4dd3-8888-36c2a1ec5a66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.012625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50d1e8-32cb-4dd3-8888-36c2a1ec5a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7f50d1e8-32cb-4dd3-8888-36c2a1ec5a66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:15.017850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:15.018711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:15.019492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:15.025226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 200/411] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:15.026814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.040801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:15.067244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f396-3a83-40b7-bb03-f72443ee95e7 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.067244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f396-3a83-40b7-bb03-f72443ee95e7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.067244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f396-3a83-40b7-bb03-f72443ee95e7 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.068664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 201/412] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:15.075472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.075746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.076016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.094191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:15.095443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:15.150653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10559a-58f8-47aa-a04b-245a3357ca86 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.175452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 202/413] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:15.200189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.200697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:15.201687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.202163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.202834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.204701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:15.205186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:15.206709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:15.217895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 212/414] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:15.224283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.361236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:15.376425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fba63e-806a-462f-a5fe-b4e405db4a02 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.402772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f45ba6-7253-49e8-9af7-8b4201717255 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.408499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:15.449449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:15.450392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.450808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.451217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.469822 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:15.471151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:15.471990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:15.477167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:15.478242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:15.479242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:15.480302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.480690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] PATH_INFO: `/v3/groups/65ba7fc74292469ca3d5d0b10e9fc0a6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.482071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] RBAC: Authorizing `identity:get_group(group_id=65ba7fc74292469ca3d5d0b10e9fc0a6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:15.483655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:15.485860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 213/415] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:15 2022] GET /identity/v3/groups/65ba7fc74292469ca3d5d0b10e9fc0a6 => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:15.487451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 203/416] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:15.500726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.502867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.680693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc1d07-87f5-4e1e-800f-955357b5a82a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.704817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84e80b-8f36-4124-9fa4-e36bbe14acb8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.747836 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:15.748492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.748782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.749048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] PATH_INFO: `/v3/projects/57e06c47f85e41698c0742550f1a32ec` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.751173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] RBAC: Authorizing `identity:get_project(project_id=57e06c47f85e41698c0742550f1a32ec)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:15.752644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:15.759102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 214/417] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:15 2022] GET /identity/v3/projects/57e06c47f85e41698c0742550f1a32ec => generated 344 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:15.764723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.780048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:15.780961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.781372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.783973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.785783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:15.788528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:15.789093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:15.790101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:15.809981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 204/418] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:15.815761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:15.873030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:15.914812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfd85e-0a10-493c-84ea-9841f696b8ee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.953423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93cdf7-b9ca-4c7d-bc4e-57938f35c483 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:15.954379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12068-814b-493e-9a59-1016acc4815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:15.955417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12068-814b-493e-9a59-1016acc4815e None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.956651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12068-814b-493e-9a59-1016acc4815e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.957126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12068-814b-493e-9a59-1016acc4815e None admin] PATH_INFO: `/v3/projects/57e06c47f85e41698c0742550f1a32ec/groups/65ba7fc74292469ca3d5d0b10e9fc0a6/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.959231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12068-814b-493e-9a59-1016acc4815e None admin] RBAC: Authorizing `identity:create_grant(project_id=57e06c47f85e41698c0742550f1a32ec, group_id=65ba7fc74292469ca3d5d0b10e9fc0a6, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:15.961260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12068-814b-493e-9a59-1016acc4815e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:15.985004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 215/419] 10.222.0.49 () {58 vars in 1442 bytes} [Sun Jul 31 18:24:15 2022] PUT /identity/v3/projects/57e06c47f85e41698c0742550f1a32ec/groups/65ba7fc74292469ca3d5d0b10e9fc0a6/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:15.985004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e623-dc23-4875-98e0-feb63a4df6df None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.985004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e623-dc23-4875-98e0-feb63a4df6df None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.985004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e623-dc23-4875-98e0-feb63a4df6df None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:15.990481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 216/420] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:15.992282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:15.992562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:15.994535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.011744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:16.012439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:16.012644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:16.012944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.033215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:16.036661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:16.145490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:16.147112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:16.166334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:16.177459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 217/421] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:16.189697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:16.200834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:16.267442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 205/422] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 18:24:16.277760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:16.320850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5d4c8-b802-4855-844d-bd540ddd646e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:16.386295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645ba29-54f4-4958-9612-913c6effd1a7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:16.421608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:16.421811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:16.422307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:16.422579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:16.422849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.423688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:16.424150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:16.424700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.426225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:16.427851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:16.430181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 206/423] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:16 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:16.439645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:16.443818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:16.448155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:16.449524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:16.450394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:16.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:16.453938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 218/424] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:16.460045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:16.565912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:16.605319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09eac8-ab0c-46d7-bd49-e15d1b86cf76 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:16.634917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf002b4-4395-4c88-a89b-76aeff7243c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:16.644403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:16.645387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:16.645881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:16.646349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.648021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:16.649760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:16.657315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 207/425] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:16 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:16.672892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:16.682048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:16.682887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:16.683365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:16.683772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.684641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:16.685105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:16.686738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:16.695998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 219/426] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:16.701197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:16.809125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c96e1-659c-4f05-b61b-91f75af8faae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:16.845163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b1de6-82d1-4ad0-bfaa-8bfed637c06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:16.846158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1de6-82d1-4ad0-bfaa-8bfed637c06e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:16.846629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1de6-82d1-4ad0-bfaa-8bfed637c06e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:16.847102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1de6-82d1-4ad0-bfaa-8bfed637c06e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:16.850631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:16.902356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:16.921294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b1de6-82d1-4ad0-bfaa-8bfed637c06e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:16.923440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b1de6-82d1-4ad0-bfaa-8bfed637c06e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:16.959647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39985297-2d95-47ff-90ba-df15ad5058be None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.005106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.006012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.006873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.007264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.008182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=2c2adfb93d804882ae83c5e3723892e1) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:17.008631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:17.010374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:17.025878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 208/427] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:16 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c31497969e7944d1ae71b619ace9c773 => generated 611 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:17.030188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 220/428] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:17.036131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.074914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80370dad-028a-418a-bcf5-766671299a27 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.075430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80370dad-028a-418a-bcf5-766671299a27 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.075906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80370dad-028a-418a-bcf5-766671299a27 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.077760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 221/429] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:17.083379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.083960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.084442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.111010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:17.112816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:17.133149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:17.208785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97747-1c1c-4ee0-8035-d8a3cd87ab58 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.246114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 222/430] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:17.263581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f879c-a135-41ad-a384-a49a3cfd41d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.264240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f879c-a135-41ad-a384-a49a3cfd41d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.264521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f879c-a135-41ad-a384-a49a3cfd41d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.264813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f879c-a135-41ad-a384-a49a3cfd41d6 None admin] PATH_INFO: `/v3/projects/2c2adfb93d804882ae83c5e3723892e1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.266013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f879c-a135-41ad-a384-a49a3cfd41d6 None admin] RBAC: Authorizing `identity:get_project(project_id=2c2adfb93d804882ae83c5e3723892e1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:17.267453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f879c-a135-41ad-a384-a49a3cfd41d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:17.269444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 209/431] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/projects/2c2adfb93d804882ae83c5e3723892e1 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:17.275945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.281765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005169-777f-47b0-b2fd-25e753b51db0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.282279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005169-777f-47b0-b2fd-25e753b51db0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.282588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005169-777f-47b0-b2fd-25e753b51db0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.284897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 210/432] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:17.285611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15005169-777f-47b0-b2fd-25e753b51db0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.288671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:17.410814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb2c6-e81d-4e23-b617-a56faa8bba7c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.422547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15005169-777f-47b0-b2fd-25e753b51db0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.445366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eae068-3e1a-4d06-8954-5185ba00f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.446043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eae068-3e1a-4d06-8954-5185ba00f144 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.446299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eae068-3e1a-4d06-8954-5185ba00f144 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.446565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eae068-3e1a-4d06-8954-5185ba00f144 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.447732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eae068-3e1a-4d06-8954-5185ba00f144 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:17.449208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eae068-3e1a-4d06-8954-5185ba00f144 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:17.455516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 223/433] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:17.455516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.455516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.455516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.464916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.465605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.465884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.466155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.470190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:17.471373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:17.477607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:17.483004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:17.483846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 18:24:17.484673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:17.485338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:17.489718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 211/434] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:17.490570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.548805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 224/435] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:17.548805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.557149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:17.597003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0cde-e952-4df7-8f99-5a8e72f6f4e7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.632615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.633281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.633560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.633832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.634627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=2c2adfb93d804882ae83c5e3723892e1) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:17.634928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:17.636376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:17.642554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 212/436] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=2c2adfb93d804882ae83c5e3723892e1&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:17.645912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.667266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8efd0-b801-4906-8c60-6bb1534a9e1b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.711786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.712477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.712782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.713077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.713824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:17.714126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:17.715571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:17.728304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 225/437] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:17.734345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.753568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:17.787730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc7f35-5bf8-445b-9e72-1b8ee5ea3715 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.822104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:17.835962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.835962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.835962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.835962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:17.849797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:17.855312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:17.856202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:17.857169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 213/438] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:17.863573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:17.878672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ae62e-7b40-44d8-ba4e-0d213c3e2d03 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:17.922487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:17.923039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:17.923373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:17.923646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.002620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2cbe-c6d8-4c75-990c-7ee5dbebfbcf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.018448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.020657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.035378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.048604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.049308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.049531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.049798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.050645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:18.050949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.052322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.061958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 214/439] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:18.067567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.137483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.145510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 226/440] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:18.151512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.203687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5dfee-a92c-4480-ab43-9adb4eb2aa66 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.251540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6753ea-ab32-412c-a174-0bd33a66a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.252178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6753ea-ab32-412c-a174-0bd33a66a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.252453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6753ea-ab32-412c-a174-0bd33a66a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.252720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6753ea-ab32-412c-a174-0bd33a66a5e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.287148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae71cba-3ad8-40ca-a05e-75268020093e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.301709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.331355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d0507-24bc-4ec5-a7c9-cd99d30af5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.332005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d0507-24bc-4ec5-a7c9-cd99d30af5df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.332279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d0507-24bc-4ec5-a7c9-cd99d30af5df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.332546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d0507-24bc-4ec5-a7c9-cd99d30af5df None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.333276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6753ea-ab32-412c-a174-0bd33a66a5e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.333730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d0507-24bc-4ec5-a7c9-cd99d30af5df None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.334736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6753ea-ab32-412c-a174-0bd33a66a5e8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.335147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d0507-24bc-4ec5-a7c9-cd99d30af5df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.337049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 227/441] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:18.337940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca084b-0c28-4340-8b56-71894d5d92f7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.338214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca084b-0c28-4340-8b56-71894d5d92f7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.338480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca084b-0c28-4340-8b56-71894d5d92f7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.339664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 228/442] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:18.343165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca084b-0c28-4340-8b56-71894d5d92f7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.385000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.435509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 215/443] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:18.436748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.437064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.437337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.445944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca084b-0c28-4340-8b56-71894d5d92f7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.454194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:18.455322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:18.479942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.480566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.480866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.481260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.482432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.483861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.485523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 229/444] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:18.486258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.524540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 216/445] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:18.525419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.532504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.587214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ef21d-eb6f-44ab-b237-7f87e7ae9eb6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.600411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.620244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.620910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.621180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.621444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] PATH_INFO: `/v3/projects/5cf7e744c64f475c8459991b467447db` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.621708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea858fa7-83e3-472c-8ae5-f753dafe92de None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.622572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] RBAC: Authorizing `identity:get_project(project_id=5cf7e744c64f475c8459991b467447db)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.624009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.625737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 230/446] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/projects/5cf7e744c64f475c8459991b467447db => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:18.626474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.657640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.658291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.658568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.658842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.659633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436, scope.project.id=5cf7e744c64f475c8459991b467447db) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:18.659936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.661480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.667383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 217/447] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:18.668154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.733348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246fe6-7a29-4103-9de7-5135b02f253f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.752957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.773081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.773751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.774020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.774359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.785921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:18.791109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:18.792052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:18.792908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:18.797453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 231/448] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:18.798434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.800862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b5ed1-8947-464a-9d63-537493dd60b2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.842609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.843259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.843591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.843853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] PATH_INFO: `/v3/groups/b52c30f30f12459ca06084d251cc43b6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.845289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] RBAC: Authorizing `identity:get_group(group_id=b52c30f30f12459ca06084d251cc43b6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.847481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.848342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 218/449] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/groups/b52c30f30f12459ca06084d251cc43b6 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:18.855920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:18.870543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:18.925911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4d834-8f39-4b85-ba98-e0512de1bd0a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.970903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:18.971995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:18.971995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:18.972177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:18.972754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:18.973140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:18.974519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:18.979017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeed69-498f-4cee-9ee2-091a34f0b092 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:18.989000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 232/450] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:18.991104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:19.023894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:19.024527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:19.024884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:19.025157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:19.025943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] RBAC: Adding query filter params (group.id=b52c30f30f12459ca06084d251cc43b6, role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=5cf7e744c64f475c8459991b467447db) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:19.026246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:19.027669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:19.034461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:19.036425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 219/451] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/role_assignments?group.id=b52c30f30f12459ca06084d251cc43b6&scope.project.id=5cf7e744c64f475c8459991b467447db&role.id=c31497969e7944d1ae71b619ace9c773 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:19.104710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa13f69-874b-4f72-a28f-6a252540116b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:19.142464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:19.143102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:19.143380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:19.143652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:19.214778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:19.216281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:19.253842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:19.319337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 233/452] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 18:24:19.324998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:19.405584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:19.430752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e60ba-86e0-4b93-a8a4-db739717df9c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:19.464935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:19.465561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:19.465836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:19.466101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] PATH_INFO: `/v3/projects/57e06c47f85e41698c0742550f1a32ec` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:19.467372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] RBAC: Authorizing `identity:get_project(project_id=57e06c47f85e41698c0742550f1a32ec)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:19.468805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:19.470567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 220/453] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:24:19 2022] GET /identity/v3/projects/57e06c47f85e41698c0742550f1a32ec => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:19.477126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:19.592963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b8876-0227-4ba1-bc5a-8a4166981b61 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:19.630803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2af660-3fb6-484f-9807-9b573080bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:19.632061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af660-3fb6-484f-9807-9b573080bd47 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:19.632446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af660-3fb6-484f-9807-9b573080bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:19.632826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af660-3fb6-484f-9807-9b573080bd47 None admin] PATH_INFO: `/v3/groups/65ba7fc74292469ca3d5d0b10e9fc0a6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:19.634795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2af660-3fb6-484f-9807-9b573080bd47 None admin] RBAC: Authorizing `identity:get_group(group_id=65ba7fc74292469ca3d5d0b10e9fc0a6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:19.637147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2af660-3fb6-484f-9807-9b573080bd47 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:19.639691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 234/454] 10.222.0.49 () {56 vars in 1180 bytes} [Sun Jul 31 18:24:19 2022] GET /identity/v3/groups/65ba7fc74292469ca3d5d0b10e9fc0a6 => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:19.645906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:19.750677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620b536-cfe5-4e88-a1c4-cc260c9824fb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:19.785856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:19.786444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:19.786719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:19.786985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:19.787774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] RBAC: Adding query filter params (group.id=65ba7fc74292469ca3d5d0b10e9fc0a6, role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=57e06c47f85e41698c0742550f1a32ec) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:19.788081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:19.789520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f3c9e-dcd7-490d-b658-3150e62bb804 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:19.795351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 221/455] 10.222.0.49 () {56 vars in 1376 bytes} [Sun Jul 31 18:24:19 2022] GET /identity/v3/role_assignments?group.id=65ba7fc74292469ca3d5d0b10e9fc0a6&scope.project.id=57e06c47f85e41698c0742550f1a32ec&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:22.138945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a1aa-21d3-402a-90d5-76866c113720 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.139471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a1aa-21d3-402a-90d5-76866c113720 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.139471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a1aa-21d3-402a-90d5-76866c113720 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.141158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 235/456] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:22.146852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4cd2d-ff90-4d83-b56c-de2259ebd31d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.147286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4cd2d-ff90-4d83-b56c-de2259ebd31d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.147377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4cd2d-ff90-4d83-b56c-de2259ebd31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.162982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:22.174058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b8f4cd2d-ff90-4d83-b56c-de2259ebd31d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:22.175805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b8f4cd2d-ff90-4d83-b56c-de2259ebd31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:22.275182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 222/457] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:22.314260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a1aa-21d3-402a-90d5-76866c113720 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:22.318585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:22.452724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a1aa-21d3-402a-90d5-76866c113720 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:22.478368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7744e-cc63-4a24-b235-32ece08d81a8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.478671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7744e-cc63-4a24-b235-32ece08d81a8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.478918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7744e-cc63-4a24-b235-32ece08d81a8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.480512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 223/458] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:22.490241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.490522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.490786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.491869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:22.492522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.492822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.493100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.504990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:22.506822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:22.521721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:22.522801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:22.523863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:22.523863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:22.523863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:22.523863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:22.525688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 236/459] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:22.531494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:22.541773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:22.542976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:22.609120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:22.681220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 224/460] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:22.691150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871c3ed-f436-4fa0-87e2-c03ca2803df3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:22.717118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:22.745557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:22.746464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.746876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.747280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.748104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:22.748938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:22.752369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26899-d773-4d89-946d-ad30a87bf012 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:22.777678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 237/461] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:22.783338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a0eac-2d02-4f91-99ee-c3e92cd19a73 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.783338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a0eac-2d02-4f91-99ee-c3e92cd19a73 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.783338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a0eac-2d02-4f91-99ee-c3e92cd19a73 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.783338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 238/462] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:22.784718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a0eac-2d02-4f91-99ee-c3e92cd19a73 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:22.820738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:22.865978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfeb24-c588-460f-af09-baf0f67b2a36 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:22.897367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a0eac-2d02-4f91-99ee-c3e92cd19a73 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:22.907601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:22.908282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.908952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.909226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.914673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:22.922693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:22.924148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:22.935036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580f351-c020-4fd4-93ea-8e5b9ece2b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:22.935652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f351-c020-4fd4-93ea-8e5b9ece2b0e None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.935921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f351-c020-4fd4-93ea-8e5b9ece2b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.936182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f351-c020-4fd4-93ea-8e5b9ece2b0e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.936884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580f351-c020-4fd4-93ea-8e5b9ece2b0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:22.938326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580f351-c020-4fd4-93ea-8e5b9ece2b0e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ff2d3f72-5d6d-446a-98d8-5c68cddd73cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:22.939738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:22.940550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:22.941446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:22.941446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:22.941446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:22.941446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:22.941446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 225/463] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:22.942267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634955-1f6a-4bf6-b587-a8213eb9b5dd None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.942364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634955-1f6a-4bf6-b587-a8213eb9b5dd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.942591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634955-1f6a-4bf6-b587-a8213eb9b5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.963309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ff634955-1f6a-4bf6-b587-a8213eb9b5dd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:22.964492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff634955-1f6a-4bf6-b587-a8213eb9b5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:22.966926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 239/464] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:24:22 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:24:22.968601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc6c0-0168-4795-bceb-2bbe7722ce92 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:22.969156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc6c0-0168-4795-bceb-2bbe7722ce92 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:22.969660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc6c0-0168-4795-bceb-2bbe7722ce92 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:22.971539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 240/465] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:22.972587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afc6c0-0168-4795-bceb-2bbe7722ce92 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.059731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 226/466] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:23.066753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.067309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.067777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.101497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:23.103079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:23.115887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afc6c0-0168-4795-bceb-2bbe7722ce92 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.152587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.154364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.154471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.154661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.155351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:23.155709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.157255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.162612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:23.177083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 241/467] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:23.178306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.195506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 227/468] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:23.195506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.228298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:23.305834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3ea46-d5c6-4a0d-9989-98282ffa9eae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.343756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.344469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.344740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.345052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.349105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef7824-785c-4467-8292-3fadd7021eac None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.356240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.357764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:23.373202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:23.374071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:23.375402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:23.375402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:23.375402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:23.375402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:23.375402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 242/469] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:23.375673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.392814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.393271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.393546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.393815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.394495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.396174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.422740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 228/470] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:24:23 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:23.423675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.472573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:23.490463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:23.510704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01009150-6e2d-467c-a31e-9e982378734a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.551143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.552322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.552893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.553484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.560196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfade52-ee8b-404c-be54-327eab06f183 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.567927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.570020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-afa36e7a-0a92-4a39-9cf4-040cecd32649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:23.588860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:23.589790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:23.590649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:23.590649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:23.590649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:23.590649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:23.597850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 243/471] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:23.599659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894f0a5-293a-4af2-afaa-49ce9e5c2801 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.600182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894f0a5-293a-4af2-afaa-49ce9e5c2801 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.600673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894f0a5-293a-4af2-afaa-49ce9e5c2801 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.602455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 244/472] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:23.603443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894f0a5-293a-4af2-afaa-49ce9e5c2801 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.604479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.605385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.605798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.606210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.607486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:23.607932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.610539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf2be5-ca99-404f-86cc-7c170c9224b3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.637094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 229/473] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:23.637094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.637094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.637094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.655857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:23.657447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:23.732550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894f0a5-293a-4af2-afaa-49ce9e5c2801 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.734005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 230/474] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:23.736077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.767909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:23.775211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.775940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.776222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.776495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.777486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:23.777790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.779886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.789136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:23.795659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 245/475] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:23.796733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.855205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9518531-d097-4b27-aa55-a9736d93f6bd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.892546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.893206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.893485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.893754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.894433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.895869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:23.917359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d9a4-44ce-4316-8fee-07a7370b91ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:23.921592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 231/476] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:24:23 2022] POST /identity/v3/users => generated 249 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:23.923636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:23.992678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:23.993898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:23.994627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:23.995813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:23.996850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:23.998726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:24.019308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 18:24:24.019308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 31 18:24:24.019308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [parameters: {'id': '9a4ba76162164b70bcd811b22533323c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 31 18:24:24.019308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jul 31 18:24:24.050227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return ref.to_dict() Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 18:24:24.051196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 18:24:24.052175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 18:24:24.053067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 18:24:24.054728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 18:24:24.055547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': '9a4ba76162164b70bcd811b22533323c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:24.057159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 31 18:24:24.059114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:24.067960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 246/477] 10.222.0.49 () {60 vars in 1131 bytes} [Sun Jul 31 18:24:23 2022] POST /identity/v3/roles => generated 144 bytes in 270 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:24.075944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:24.123945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:24.145421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:24.154960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6be14-59f9-4d14-a5e3-03243043513d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:24.188499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f669e52-713c-40cb-bbf6-a404df03e442 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:24.198169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:24.199399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:24.199967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:24.200433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:24.201458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:24.203375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:24.228400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762bc0b-80ef-4ea9-bdb3-515703e222f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:24.230305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762bc0b-80ef-4ea9-bdb3-515703e222f6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:24.230305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762bc0b-80ef-4ea9-bdb3-515703e222f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:24.230305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762bc0b-80ef-4ea9-bdb3-515703e222f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:24.240419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 232/478] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:24:23 2022] POST /identity/v3/users => generated 248 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:24.243622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762bc0b-80ef-4ea9-bdb3-515703e222f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7762bc0b-80ef-4ea9-bdb3-515703e222f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:24.249150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:24.250432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 18:24:24.251433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:24.253967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 247/479] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:24:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:24.262086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:24.392293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8698b-2cec-4d5f-9def-b5c5eb46b8a7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:24.432655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:24.433311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:24.433641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:24.433912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:24.434632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:24.434926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:24.436289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482f397-86f7-49f7-bb2c-4a967d41cb7d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:24.438620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:24.446685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 233/480] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:25.316460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469be77-98fe-426c-8afd-099ff5df0ded None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:25.317047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469be77-98fe-426c-8afd-099ff5df0ded None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:25.317047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469be77-98fe-426c-8afd-099ff5df0ded None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:25.318582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 248/481] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:25.323925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:25.324342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:25.324561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:25.349705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:25.350898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:25.439104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 234/482] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:25.472284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469be77-98fe-426c-8afd-099ff5df0ded None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:25.590895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469be77-98fe-426c-8afd-099ff5df0ded None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:25.612885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:25.631573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:25.632240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:25.632522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:25.632819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:25.649908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:25.655218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:25.656083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:25.657055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:25.659810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 249/483] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:25.665796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:25.799686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fa05a-1bbd-4839-9d6b-5536c732eea3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:25.837689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:25.847027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:25.847881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:25.848345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:25.848842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:25.849780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:25.850275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:25.851878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:25.863738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 235/484] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:25.869527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:25.994643 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77dbd2-1850-4dc0-9783-0cdaaec7c04f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:26.043097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:26.044118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.044600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.045079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.069682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:26.071489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-afde4383-184d-4fd1-8068-a08ff0679225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:26.098173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:26.099080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:26.099897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:26.099897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:26.099897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:26.099897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:26.103524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 250/485] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:26.108848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:26.245237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941a602-ffc9-419d-90d7-aa5a9b7eee87 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:26.291361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3639d-dee0-4522-8ea6-625fb933b783 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.291699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3639d-dee0-4522-8ea6-625fb933b783 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.291886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3639d-dee0-4522-8ea6-625fb933b783 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.293387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 251/486] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:26.301160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:26.301787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.302061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.302329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.303029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:26.303328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:26.304691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:26.309147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.309656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.310119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.319824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 236/487] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:26.325788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:26.346516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:26.349516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:26.409115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:26.409553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:26.470599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f1e6e-b63f-4bd3-adef-19eaf1720c87 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:26.513069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 252/488] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:26.536159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:26.537168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.537656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.538112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.550665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:26.552411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:26.570005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:26.570926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:26.571787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:26.577483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:26.577906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 237/489] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:26.584973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:26.705835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a77bd-deb8-4e7f-846f-b655089de6c2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:26.709743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c30a9c-a10b-415f-a0f3-84006132e0b0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:26.720363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:26.741532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcd7dd-553c-403f-b9e8-c0233e8731b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:26.742184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcd7dd-553c-403f-b9e8-c0233e8731b8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.742467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcd7dd-553c-403f-b9e8-c0233e8731b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.742721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcd7dd-553c-403f-b9e8-c0233e8731b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.747808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:26.748421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.748722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.749023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.749828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:26.750165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:26.751632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644b3f1-a7df-41ad-92ee-5308e19a6a30 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:26.753221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dcd7dd-553c-403f-b9e8-c0233e8731b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a0dcd7dd-553c-403f-b9e8-c0233e8731b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:26.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:26.759572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 253/490] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:26.765809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0415f5-0614-4a26-a2b0-77024523845f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.766527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0415f5-0614-4a26-a2b0-77024523845f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.766527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0415f5-0614-4a26-a2b0-77024523845f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.766527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 254/491] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:26.766527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf570-8f08-41dc-8a28-43bf373c32c2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.766527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf570-8f08-41dc-8a28-43bf373c32c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.766527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf570-8f08-41dc-8a28-43bf373c32c2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.768753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 255/492] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:26.769733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddf570-8f08-41dc-8a28-43bf373c32c2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:26.796843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 238/493] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:26.796843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57e5e-1c26-4fee-bb27-b9573cfb3ed9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.796843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57e5e-1c26-4fee-bb27-b9573cfb3ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.796843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a57e5e-1c26-4fee-bb27-b9573cfb3ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.803947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:26.813966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-02a57e5e-1c26-4fee-bb27-b9573cfb3ed9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:26.815176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-02a57e5e-1c26-4fee-bb27-b9573cfb3ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:26.882729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddf570-8f08-41dc-8a28-43bf373c32c2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:26.886395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 239/494] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:26.887842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.888361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.888858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.907536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:26.909180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:26.929670 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:26.930351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:26.930632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:26.930905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:26.931624 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:26.931926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:26.933368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:26.942238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 256/495] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:26.942238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:26.983522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 240/496] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:26.984286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.016026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.048259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.064892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523381-3ae1-4519-bd27-fbe0d4c5873c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.089253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246ef79-b634-477c-8302-f48aa1ed5eb2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.097323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.097937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.098212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.098516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.125262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.125895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.126172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.126437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.137080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:27.142156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:27.143170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:27.143960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 241/497] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:27.144602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.169240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.170728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.235057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.248006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66b0c8-9dd6-4876-9ede-d0e6093516f5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.254137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.271405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 257/498] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:27.272272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.286017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.286657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.286927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.287193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.297343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.298787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-fe9fade7-47d5-4b47-8e2a-a160200d8fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:27.313514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:27.314362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:27.315231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:27.315231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:27.315231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:27.315231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:27.315231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 242/499] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:27.315928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75791a-cab3-4ab6-9387-5ad0bc53fc83 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.316015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75791a-cab3-4ab6-9387-5ad0bc53fc83 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.316274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75791a-cab3-4ab6-9387-5ad0bc53fc83 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.317638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 243/500] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:27.320803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75791a-cab3-4ab6-9387-5ad0bc53fc83 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.371419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0444762-e056-4a5b-9e28-7e6ee8f61429 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.405543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.406324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.406603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.406925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.417197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:27.418237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75791a-cab3-4ab6-9387-5ad0bc53fc83 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:27.422257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:27.423144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:27.423921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 258/501] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:27.424576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.453322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.453946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.454219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.454477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.455199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:27.455519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.456947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.466646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 244/502] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:27.467819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.470267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.470704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.525117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711253d-c3db-4aec-9b11-14540837c982 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.559797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.560432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.560689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.561009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.569954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e0ff3-9974-4dd1-8eb9-62173fe3adef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.570957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.572400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-170a58e9-b401-4c62-a697-8a21212db7a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:27.586912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:27.587722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:27.588572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:27.588572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:27.588572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:27.588572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:27.588572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 259/503] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:27.589193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.589465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.589728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.607079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.607696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.607849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:27.607944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.608223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.608935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:27.609027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:27.609484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.610862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.621958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 245/504] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:27.622695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.675018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 260/505] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:27.675854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.694924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.703368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.735915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4869d5-27db-468e-be84-1e508625bb2e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.770040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.770661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.770939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.771209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.771924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:27.772217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7b24-3a48-4d4e-b159-8b4549747f3b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.772306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.773629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.781657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 246/506] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:27.782374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.805816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.806450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.806723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.806991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.817191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.818683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:27.834073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:27.834948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:27.835775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:27.835775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:27.835775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:27.835775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:27.835775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 261/507] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:27.836265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.884778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5599ca-8d73-4d06-a583-48efb7497e96 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.907698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.916315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:27.919633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.920247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.920519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.920809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.921496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:27.921815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.923173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.934563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 247/508] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:27.935330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:27.939667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35233d10-1ee9-4c49-8fef-4c47efa257fe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:27.973881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:27.974925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:27.975199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:27.975439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:27.983367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:27.984914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:27.997450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:27.998507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:27.999413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 262/509] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:28.000029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.038072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f700-ab40-480c-ae36-a1fb1c75e05c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.071393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.072024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.072295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.072558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.082305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.083712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.097901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cfbcb-d711-4abb-8e31-415f53fc9827 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:28.098371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:28.099312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:28.100135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:28.100135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:28.100135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:28.100135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:28.100135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 248/510] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:28.100402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.134409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.136842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.137587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.137861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.138124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.145062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.149038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.150640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:28.164959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:28.165924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:28.171306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:28.171306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:28.171306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:28.171306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:28.171306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 263/511] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:28.171306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.199977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c472c-3295-4f6c-abf8-8721e0bd92cc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.235358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.235976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.236246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.236509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.237229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:28.237522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.238888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.250030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 249/512] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:28.250744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.271616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d8c3e-864a-4350-b9a0-ed6cf8b7e4b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.304384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.304989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.305263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.305532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.311990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.313490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:28.325714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:28.326710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 264/513] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:28.327743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.345843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.351728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e6c9f-e6e5-4321-a098-b56c9dc5d472 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.380689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.385203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.385819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.386095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.386370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.387168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:28.387469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.388987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.406082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 250/514] 10.222.0.49 () {56 vars in 1137 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:28.407016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.431881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f41f0-032e-44c8-bc8b-090227fbd3ca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.464458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.465103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.465373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.465636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.466323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:28.466617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.467980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.479391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 265/515] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:28.480158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.504086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1126a-6a1e-48a1-a773-88b864974ebb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.537192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.537801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.538078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.538343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.539024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:28.539318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.540664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.551428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 251/516] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:28.552201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.568374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.580197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549403f-dc6e-43fa-b82e-fa103f059003 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.590622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.613989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306dd4-a10e-425b-9142-454f019692be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.614615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306dd4-a10e-425b-9142-454f019692be None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.614894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306dd4-a10e-425b-9142-454f019692be None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.615160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48306dd4-a10e-425b-9142-454f019692be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.622213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48306dd4-a10e-425b-9142-454f019692be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.623686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48306dd4-a10e-425b-9142-454f019692be None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-48306dd4-a10e-425b-9142-454f019692be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:28.635741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:28.636632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:28.637533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 266/517] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:28.638086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306dd4-a10e-425b-9142-454f019692be None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.654660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78f334-2fd4-4685-93d0-1c002225549f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.687391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.688159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.688501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.688864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.689643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:28.689932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.691344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.708698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 252/518] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:28.709492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.734357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48306dd4-a10e-425b-9142-454f019692be None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.766961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.767574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.767851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.768125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.786557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.806677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:28.811669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f5749-fc00-42b5-8e7f-4127e81df11b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:28.836708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.838166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.847648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:28.848268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:28.848543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:28.848813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:28.849467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:28.850884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:28.873916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 253/519] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:24:28 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:28.874716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.970313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 267/520] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:28.970313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:28.990485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c3b3c-898c-47ce-a8e9-944ee3aee7ac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.025785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.034978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.035960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.036378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.036650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.046746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.047813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.053646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:29.059675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:29.060740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:29.061792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 254/521] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:29.062507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.096865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad819333-1318-4fa2-9a27-8493696630e6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.129146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.129779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.130037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.130301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.131091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:29.131778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.133249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.167733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d309c-fbac-400e-96a2-198b4d12da11 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.173573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 268/522] 10.222.0.49 () {56 vars in 1139 bytes} [Sun Jul 31 18:24:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:29.173573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.207522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.208145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.208418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.208687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.209502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=9958737194be4488a1e09e6e48557d75) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:29.209805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.211650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.223312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 255/523] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/role_assignments?user.id=9958737194be4488a1e09e6e48557d75&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:29.224553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.289186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.295551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576a64-3555-43d1-8e5a-a69867c836a9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.328581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.338881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.339941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a371474-5a79-4f9f-9659-e862694b1cf8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.340174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.340633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.341173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.354023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.356065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:29.374158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:29.375284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:29.376097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:29.376097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:29.376097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:29.376097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:29.377170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.377811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.378084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.378349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.379090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 269/524] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:29.379195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:29.379431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.380520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.386536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.412699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 256/525] 10.222.0.49 () {56 vars in 1137 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:29.413941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.504691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1a72-4784-4834-9576-e6f194700432 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.517158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210437-c44f-43dc-ac34-e7c7d07e5c52 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.531622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.540947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.541774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.542112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.542477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.555689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.556545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.557027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.557432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.558260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:29.558737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.560291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.573000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 257/526] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:29.574088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.596337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.617273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.618628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.683006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37d92-9860-4a67-ab7b-36a0a54a36f6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.734043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 270/527] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:29.734043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.738745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.739236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.739502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.739768 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.794825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.823957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.825478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.833501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:29.858329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14140cc8-0db7-4802-bba0-975f531e41b2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:29.898544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:29.899350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:29.901876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:29.904143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:29.915778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:29.921355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:29.934841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:29.935770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:29.936607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 271/528] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:29.937469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:29.960889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 258/529] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:29.961885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.040468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:30.068621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d61985-8433-43c0-88e5-de5e9132cc19 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.077874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c679c-6d9f-4e07-a34a-fbc5fa54cc11 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.112365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.113036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.113308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.113572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.114563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.115182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.115459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.115730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.116485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:30.116825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.118266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.122623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:30.133706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.137423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.137542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 259/530] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:30.137630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:30.151423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:30.152388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:30.153253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:30.153253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:30.153253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:30.153253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:30.159609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 272/531] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:30.160647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.253529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b581e-b064-4548-bab9-d863fad1c7db None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.263174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ce639-6532-4162-a48d-5735408a0ea3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.284176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:30.291142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.291721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.292051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.292358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.299665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.300495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.300956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.301389 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.302333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=20341bc6269f477c952c25437ab7a2d1) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:30.302824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.303815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.304637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.305324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.311258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 273/532] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/role_assignments?user.id=20341bc6269f477c952c25437ab7a2d1&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:30.312393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:30.330250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:30.331545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:30.333566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:30.333566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:30.333566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:30.333566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:30.333566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 260/533] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:30.337954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.381858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:30.430295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b0f2-f157-4478-b2be-6a874a78ce44 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.467072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.468097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.468587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.469119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.469995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:30.470479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.472039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.490055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 274/534] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:30.491923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.531549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccad8c-db12-46f0-a5cf-8a515941d801 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.612992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.614002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.614496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.614774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.615465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:30.615760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.617226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.651118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 261/535] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:30.661752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.700706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7504c-005c-4be4-b819-a0fd22422ffb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.731362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:30.758604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:30.767586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.768243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.768513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.768794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.782150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.783630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:30.803148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:30.804547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:30.806065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:30.811479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 275/536] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:30.813087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:30.867226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c4900-f6d6-40ae-a4eb-f7af40a9725b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:30.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:30.937685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.937685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.937685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.947916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:30.949415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-41d51dfe-8733-4c3e-b4df-f6bc7383bfd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:30.973411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:30.974307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:30.975157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:30.977419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 262/537] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:30.979683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9eb1-cd2b-41d1-b811-c1456ab08c49 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:30.980010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9eb1-cd2b-41d1-b811-c1456ab08c49 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:30.982717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9eb1-cd2b-41d1-b811-c1456ab08c49 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:30.987415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 263/538] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:30.988959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9eb1-cd2b-41d1-b811-c1456ab08c49 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.011504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c3d28-8581-4918-86e4-9143d076871f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.060439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.061103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.061370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.061634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.062428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:31.066803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:31.070446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0972f53-9c43-45c9-aa09-23c387630ab6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:31.112012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 276/539] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:31.116660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.119125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9eb1-cd2b-41d1-b811-c1456ab08c49 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.119263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.119263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.120949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:31.153428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.154172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.154303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.155485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.155485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:31.155654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:31.157253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:31.161433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:31.162628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:31.174458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 264/540] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:31.175753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.244789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:31.288789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599da09-0bc2-481c-841a-1ea4020267f6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.325416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.326038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.326310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.326577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.327360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=f590f668ad724bf087b164bcf3f543ba) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:31.327666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:31.329017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 277/541] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:31.329354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:31.329901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.342318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 265/542] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/role_assignments?user.id=f590f668ad724bf087b164bcf3f543ba&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:31.390772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.426263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:31.469966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5d41a-d15d-47c7-a1d3-9e0c67cb8e51 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.507091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.507637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.507904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.508168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.508996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=7e84fa2255614cde84495eedcc0971a9) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:31.509384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:31.510842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7f16-40e4-4cea-a91f-ab911067cfe3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:31.516900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 278/543] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/role_assignments?user.id=7e84fa2255614cde84495eedcc0971a9&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:31.518121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a5c38-1023-4718-8d54-ada64aaf2a32 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.518393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a5c38-1023-4718-8d54-ada64aaf2a32 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.518656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a5c38-1023-4718-8d54-ada64aaf2a32 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.519775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee072c8-5149-460f-80cc-5495d218abfb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.520116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 279/544] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:31.526438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.526809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.527151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.549900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:31.551117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:31.568251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.569154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.569558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.569835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.582576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:31.587604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:31.596059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:31.598058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:31.599860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 266/545] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:31.604515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.633053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 280/546] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:31.654657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.719654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:31.734803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030661d-36ff-4331-952c-db87d2c48e58 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.793626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.794187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.794461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.794726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.795452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:31.795759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:31.798152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:31.810124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 267/547] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:31.821817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.844786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623fc02-be1b-4cd1-b544-98ef3a7c1383 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.876797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.877456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.877732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.878003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:31.898656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:31.904455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:31.905454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:31.906278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:31.910953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 281/548] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:31.916567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:31.944146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:31.956628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00e87-8978-4f74-beb2-1178bf9e923e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:31.991097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:31.991742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:31.992019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:31.992285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.003254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.004708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:32.019709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:32.021671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:32.024144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:32.024144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:32.024144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:32.024144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:32.024144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 268/549] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:32.029754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.032665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:32.051784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6ab4a-64a7-4e4e-a1d6-099a99f86a56 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.100542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.101198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.101475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.101749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.102484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:32.102786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.106400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.126000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 282/550] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:32.130225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.224744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:32.237303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa851f-565f-4fdf-900a-89ab908a50fd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.266993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb363c4-17b5-4c78-bd36-080c27d0f03c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.275548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.276170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.276442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.276708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.277429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:32.277722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.279649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.294755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 269/551] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:32.302794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.329125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.329992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.330270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.330541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.351800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.353504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:32.382093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:32.383116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:32.384057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:32.384057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:32.384057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:32.384057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:32.386452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 283/552] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:32.390346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.441187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:32.459366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc451abd-76cd-45de-8e9f-0820be8fa8e0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.496218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.496898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.497309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.497589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.505363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.507188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:32.521107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:32.522035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:32.522924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:32.523449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 270/553] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:32.528506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.636363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34209a-a036-4dec-b4fb-3bd6b0d6e0cd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.671941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.672576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.672882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.673262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.674166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:32.674479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.676036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e2ab2-1406-4e97-b5af-048fa17d0107 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.677915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:32.695704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 271/554] 10.222.0.49 () {56 vars in 1135 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:32.701078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b9ad-fe8f-4690-aaf7-8838a2791b16 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.701381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b9ad-fe8f-4690-aaf7-8838a2791b16 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.701661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388b9ad-fe8f-4690-aaf7-8838a2791b16 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.703241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 272/555] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:32.704749 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388b9ad-fe8f-4690-aaf7-8838a2791b16 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.707731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c3e7-313a-4d22-a1e3-338f5b41591d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.752436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.753350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.753625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.753893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.754820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:32.755120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.757644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f77e1c-ac9e-46fc-96f3-a0c99093679f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:32.759709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:32.775193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 284/556] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:32.776830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.777116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.777385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.802203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:32.803942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:32.817994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388b9ad-fe8f-4690-aaf7-8838a2791b16 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:32.871820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:32.872916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:32.873333 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:32.873729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:32.886482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 285/557] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:32.888158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:32.955907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:32.982910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:32.984843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.016187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682d1fa-561a-4f46-adf4-8b1244a9e3d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.046526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.062058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.062798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.063074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.063345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.073074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.074575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.087789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 273/558] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 18:24:33.088709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:33.095155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:33.097763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 286/559] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:24:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:33.099921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.177928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.199234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884a7a-ac6e-400d-bc97-f1e8e221a3d2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.212796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0274-22d0-4662-8197-ffdee3135148 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.234903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.235546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.235873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.236138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.246801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:33.251859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:33.252912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:33.253746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 274/560] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:33.254439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.264303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.265060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.265396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.265785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.280193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.282161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:33.301888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:33.303665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:33.305826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:33.305826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:33.305826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:33.305826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:33.305826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 287/561] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:33.305826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.346672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.377031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556b544-a3cd-4431-a328-866e0424f72d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.414868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.415651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.416023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.416386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.416906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf59a-60fd-4d87-a5f0-60b2981be7a0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.417443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:33.417834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.419449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.429366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.445650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 275/562] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:33.445650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.460228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.460890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.461208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.461488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.462629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:33.462933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.464590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.480082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 288/563] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:33.480082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.570589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830dd13-8937-4c68-9d9f-c038bfe20ad3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.611489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.615028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.615729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.616000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.616282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.616986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:33.617278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.618653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.622125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507b17a-8a48-4b6f-b767-a93c63efd925 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.637608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 276/564] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:33.637608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.672924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.674246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.675058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.675762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.690512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.692412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.703128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:33.707594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:33.708446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:33.710636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:33.710636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:33.710636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:33.710636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:33.713625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 289/565] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:33.714460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.751241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea1b70-6937-4db2-839a-855e81484433 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.785794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.786499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.786776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.787046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.797700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.799144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6756a0c7-37db-455c-9e9d-2a21a2d85cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:33.813012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:33.813942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:33.814807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:33.814807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:33.814807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:33.814807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:33.814807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 277/566] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:33.815266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22538f5d-951f-48c2-9b02-9f9f6b0cc3be None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.815571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22538f5d-951f-48c2-9b02-9f9f6b0cc3be None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.815809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22538f5d-951f-48c2-9b02-9f9f6b0cc3be None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.817055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 278/567] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:33.817733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22538f5d-951f-48c2-9b02-9f9f6b0cc3be None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.822952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43f61-cc75-4cab-a521-2169d2527630 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.856151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.856791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.857091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.857340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.863891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.865383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c2a6e6c4-31e9-4f48-8cba-2b30c8f87719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:33.876786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:33.877660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:33.878568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:33.880701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 290/568] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:33.882379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6c623-4fc8-4101-b63e-351624f0e5c2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.882660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6c623-4fc8-4101-b63e-351624f0e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.882935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6c623-4fc8-4101-b63e-351624f0e5c2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.884311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 291/569] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:33.885148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6c623-4fc8-4101-b63e-351624f0e5c2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:33.897728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.921469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22538f5d-951f-48c2-9b02-9f9f6b0cc3be None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.931867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:33.956558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:33.957145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.957419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.957692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.958381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:33.958676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:33.960049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f066f-e3fb-42ed-b520-bbf829b84076 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:33.971663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 279/570] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:33.971663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e572b-022f-4a97-bdad-c6c22d7163e1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:33.971663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e572b-022f-4a97-bdad-c6c22d7163e1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:33.971663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e572b-022f-4a97-bdad-c6c22d7163e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:33.987854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6c623-4fc8-4101-b63e-351624f0e5c2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:33.991022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ba4e572b-022f-4a97-bdad-c6c22d7163e1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:33.992260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ba4e572b-022f-4a97-bdad-c6c22d7163e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:34.022015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.022635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.022939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.023206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.023887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.024182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.025576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.036967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 292/571] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.037849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.064223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 280/572] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:34.065922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.066201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.066465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.093332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:34.094523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:34.105740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.136678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d529ad-8573-483d-b124-9e1b70380f92 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.158885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 281/573] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:34.159702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.170960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.171580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.171950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.172216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.172990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.173289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.174732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.195936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 293/574] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.195936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.202562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.271344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31712eac-b003-4aa0-83fd-9248bb67b065 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.291632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c5315-6e41-494d-a68b-850a43ae3858 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.304859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589489f-28e8-4839-8796-560e06930818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.305610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589489f-28e8-4839-8796-560e06930818 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.305883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589489f-28e8-4839-8796-560e06930818 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.306146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589489f-28e8-4839-8796-560e06930818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.312928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589489f-28e8-4839-8796-560e06930818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.314363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589489f-28e8-4839-8796-560e06930818 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.318954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.324916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.325548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.325820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.326081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5589489f-28e8-4839-8796-560e06930818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:34.326470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:34.327478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 282/575] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:34.328621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589489f-28e8-4839-8796-560e06930818 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.336456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.337955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:34.349313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:34.350263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:34.351208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 294/576] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:34.352498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.421821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.431583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589489f-28e8-4839-8796-560e06930818 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.454174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a422f-ac96-4c30-bf26-61907af6ae06 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.466125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.466784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.467055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.467346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.477439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:34.482245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:34.483344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 283/577] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:34.484453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.491819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.492461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.492733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.493026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.503977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:34.509669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:34.510582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:34.511407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 295/578] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:34.512091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.565087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.586739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca41a9-0834-44b1-86a5-5293e35662d7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.612184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b15c-c9e4-4cd0-8780-02e9036a097e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.620941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.621881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.622151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.622692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.623476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=e64871a43e564c9bbf7ebfdbd512a1f3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.623777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.625283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.630665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 284/579] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/role_assignments?user.id=e64871a43e564c9bbf7ebfdbd512a1f3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.631818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.649313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.655470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.656208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.656486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.657291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.657546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.657850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.659345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.679312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 296/580] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.679312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.751310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73b064-9b7c-40e5-8a0c-76e19656a034 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.789111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.789799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.790081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.790355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.790690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c4dac-bd3e-412e-b710-78b83a40a77e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.791145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.791452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.792897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.805540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.810811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 285/581] 10.222.0.49 () {56 vars in 1137 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.811910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.836581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.837231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.837503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.837847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.838471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.838764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.840298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.849067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 297/582] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.849398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.891517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:34.922324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697579fa-d859-4aa8-81bb-f5a5425b4c51 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.952710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158864a-6c9f-4d1a-ab0b-f2de9bf85fc9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:34.959361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.960388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.960668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.961012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:34.962084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:34.962387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:34.964446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:34.979414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 286/583] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:34.979414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:34.986114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:34.986779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:34.987049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:34.987314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/9958737194be4488a1e09e6e48557d75/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.001945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=9958737194be4488a1e09e6e48557d75, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.003323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.017615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 298/584] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:34 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/9958737194be4488a1e09e6e48557d75/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:35.017615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.061226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.107840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc2336-c868-4389-ba6b-b112aa7f51fe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.146915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f53bc-0b5b-47bc-a25b-23cbe1e0ad28 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.152888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.153669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.154006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.154340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.157410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.170079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.171727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.190088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.190788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:35.191046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:35.192020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:35.192904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:35.192904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:35.192904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:35.192904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:35.193138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.193138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.196960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 287/585] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:35.198054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.206625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.208364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:35.233536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:35.234534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:35.235407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:35.235407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:35.235407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:35.235407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:35.235407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 299/586] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:35.236095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.316118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.330041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd631e-bd9f-472b-af7a-1ef4ba6a3b38 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.343764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51695f7e-6d4f-420e-85d3-fb08d0837ed0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.377047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.377692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.377965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.378449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.384834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.385741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.386206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.386663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.388083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:35.388578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.390895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.391026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.392322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:35.410285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:35.411212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:35.412080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:35.412080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:35.412080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:35.412080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:35.412080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 288/588] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:35.412080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.413172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 300/588] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:35.414243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.453856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.520370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae1ae-4fef-495b-b5e8-2a86562077f1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.535013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6f201-c493-40d2-aa0b-f74282468bd1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.565078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.566424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.566952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.567460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.568547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:35.569119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.569744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.570458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.570788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.571125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.571480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.573351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.579208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.580851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.590270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 289/589] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:35.591319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:35.600631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:35.601595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:35.602453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:35.606893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 301/590] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:35.607853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.704786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.704786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a9dcc-734d-4522-b2f2-a7fe5e9d0710 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.727834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de575-83fa-471f-89ec-250f78c1529b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.737305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.738042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.738421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.738760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.739545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:35.739928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.741456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.759396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 290/591] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:35.760413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.775852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.776564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.777119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.777395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.787590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.789744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:35.805612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:35.806730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 302/592] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:35.807637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.868084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.893286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916ad5d-cd61-4375-8b69-bd368ce10a56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.926799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbe0e1-f5d0-4372-a60e-69f1e94f6272 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:35.929426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.930187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.930580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.930947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.931791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:35.932191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.933900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.952006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 291/593] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:35.953196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:35.964797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:35.965802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:35.966088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:35.966357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:35.972949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:35.976958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:35.979003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:35.993854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:35.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 303/594] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:35.997650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.066177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbab1-b3bb-485c-a784-3da98c212b50 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.102423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.103401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.104119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.104468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.105424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:36.105791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.107826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.122570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c0967-cfbe-4401-8497-7776d46f03ab None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.131032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 292/595] 10.222.0.49 () {56 vars in 1139 bytes} [Sun Jul 31 18:24:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:36.133660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.140664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:36.170093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.171074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.171354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.171677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/20341bc6269f477c952c25437ab7a2d1/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.192306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=20341bc6269f477c952c25437ab7a2d1, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.194346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.214697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 304/596] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:35 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/20341bc6269f477c952c25437ab7a2d1/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:36.216724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.237645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:36.293684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba33b2-8eac-413d-91d8-596eb0bda69d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.348240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.349970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.350603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.351644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.352896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:36.354091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.355895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.400987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 293/597] 10.222.0.49 () {56 vars in 1137 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:36.406321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.433375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21143821-3aa2-4251-a1b1-c205d75c9d2b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.474075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.474728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.475000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.475324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.486383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.487984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:36.505106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:36.506634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:36.507851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:36.507851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:36.507851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:36.507851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:36.507851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 305/598] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:36.512788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.532429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:36.536692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e2296-f8e3-4b3c-b924-4bd597020b82 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.580704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.581634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.582323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.584231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.587433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:36.606191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.607669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:36.637971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:36.639108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:36.640017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:36.640017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:36.640017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:36.640017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:36.640017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 294/599] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:36.645952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.674788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25a698-2f46-47e4-a8f2-bf04059270e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.711940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.712573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.712890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.713168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.713869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:36.714255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.715678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30f3df-3382-49bf-b82d-d08e161d9f79 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 306/600] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:36.730419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e59b8-da31-40dc-a57c-7b207c82e272 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.730844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e59b8-da31-40dc-a57c-7b207c82e272 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.731114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e59b8-da31-40dc-a57c-7b207c82e272 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.733164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 307/601] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:36.737895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e59b8-da31-40dc-a57c-7b207c82e272 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.789187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc11e5-051c-45bb-9047-c44a26c71939 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.831537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.832249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.832710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.833070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.834592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:36.834977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.836429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81741e-ce10-4f61-b699-2df1b5a74b07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.840185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:36.855653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 295/602] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:36.857121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.857470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.857852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.865694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e59b8-da31-40dc-a57c-7b207c82e272 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:36.876841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:36.878345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:36.902922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:36.903623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:36.903901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:36.904274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:36.912068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:36.913710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:36.924729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:36.926579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:36.927432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:36.928312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:36.930889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 308/603] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:36.932028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:36.960935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 296/604] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:36.962037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.053736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544740e-5c48-498f-ad63-509cdee83550 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.086504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c765-dcdb-4394-8cc8-2daa6b2fd066 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.088398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.089082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.089355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.089617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.092603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:37.097570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.100001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:37.113430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:37.114356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:37.115238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 309/605] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:37.115721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.130133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.130735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.131043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.131670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.132421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:37.132729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.134183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348f6b0-02de-43cf-b913-0953b33e8ff0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.151237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 297/606] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:37.152488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ca9e-b0fd-4e49-85ff-220253b96712 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.152751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ca9e-b0fd-4e49-85ff-220253b96712 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.153062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ca9e-b0fd-4e49-85ff-220253b96712 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.158324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 298/607] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:37.158324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ca9e-b0fd-4e49-85ff-220253b96712 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.197671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:37.230199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1bb0-23e2-4799-ab4f-16e464660f0b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.265944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.266582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.266863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.267134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.270425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ca9e-b0fd-4e49-85ff-220253b96712 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.277413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:37.282789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:37.283969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:37.284808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 310/608] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:37.285479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.314159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.314772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.315089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.315355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.316121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:37.316419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.317854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df0faf-bfff-4b4c-bc95-9ef31fd302eb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.330536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 299/609] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:37.331705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.331985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.332255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.350258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:37.350577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:37.352062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:37.398018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fde85-79b7-4c3d-a324-ed91044206d8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.421577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 300/610] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:37.422654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.436480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.437315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.437660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.438015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/f590f668ad724bf087b164bcf3f543ba/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.453798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=f590f668ad724bf087b164bcf3f543ba, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.455188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.463312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:37.471383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 311/611] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:37 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/f590f668ad724bf087b164bcf3f543ba/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:37.472420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.569341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c9c0b-7d16-4633-9216-4d4c38633aa5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.591516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5d59b-22b5-4651-80dd-8f3c2169c7c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.601026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:37.618522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.619638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.621156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.621503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.622329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:37.622697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.624888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.638896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 301/612] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:37.638896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.644314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.645932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.646214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.646492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/7e84fa2255614cde84495eedcc0971a9/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.664888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=7e84fa2255614cde84495eedcc0971a9, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.666668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.685074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 312/613] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:37 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/7e84fa2255614cde84495eedcc0971a9/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:37.686029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.743465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:37.766733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360372ea-8846-4890-a830-e96cf3c3d314 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.824048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef479ed-ce4a-47dc-9920-dd2613e312e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:37.863058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.863811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.864093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.864373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.870078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:37.870743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:37.871020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:37.871291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:37.876947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:37.878840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:37.898344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:37.899450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:37.900463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:37.900463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:37.900463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:37.900463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:37.900463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 313/614] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:37.902043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:37.906480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:37.907506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:37.908424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:37.909917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 302/615] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:37.915126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:37.919191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.014267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f00c89-251f-4e4a-9402-4e846a89d722 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.052973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.052973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.052973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.052973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.053420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:38.053420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.053420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.057853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d843aeb-c91f-4683-86ed-19d6672c170e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.072797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 314/616] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:38.072797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.141274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.154896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.155525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.155803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.156303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.157047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:38.157344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.159128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.172211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 303/617] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:38.188103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.203620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.214844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9f5e-c97c-4d51-94dc-5009dbb25158 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.281036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.281629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.281905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.282175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.292649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.294181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:38.320366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:38.321619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:38.322772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:38.329468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 315/618] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:38.336680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.350042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad20cb5-fab3-4ed9-b1b7-96efeaf55102 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.386228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.386875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.387145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.387409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.398637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.400240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:38.416104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:38.417269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:38.418469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:38.418469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:38.418469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:38.418469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:38.418469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 304/619] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:38.429692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.495054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbb588-7e52-4b77-8cf6-3e51b3a4f690 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.529155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.533893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.534530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.534808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.535076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.535936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:38.536237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.537792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.554830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.571590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 316/620] 10.222.0.49 () {56 vars in 1135 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:38.580518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.631982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352cbe-e594-468c-b0c3-9b4fff559fac None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.685133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.685760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.686038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.686321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.687009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:38.687344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.688752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf332d-7a0d-4ca9-9201-d41495eeb61b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.705953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 305/621] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:38.707302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ee02e-ae86-4954-8a05-72859d073fca None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.707585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ee02e-ae86-4954-8a05-72859d073fca None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.707856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ee02e-ae86-4954-8a05-72859d073fca None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.713802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 306/622] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:38.715119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.715405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.715672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.720256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc127fea-91da-43a1-827a-1138404af554 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.734889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:38.736097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:38.754823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.755485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.755754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.756020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.767985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.769551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:38.785401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:38.786337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:38.787221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:38.787221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:38.787221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:38.787221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:38.787221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 317/623] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:38.787915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.817033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 307/624] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:38.818017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.836437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.899940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc380a-b486-4bb4-94ea-4a1b174535f8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.910883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:38.937298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.938042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.938675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.938675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.946409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.947921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:38.951943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0165f-64c6-4f3e-ae88-c100cf9a50c1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:38.959417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:38.960552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:38.961743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:38.967304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 318/625] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:38.968190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:38.991533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:38.992461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:38.992983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:38.993548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:38.994399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:38.994900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:38.996670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.014325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 308/626] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:39.015649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.072320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.076088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec40753-9dec-416a-abeb-a6cd364d5ded None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.112107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.112666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.113040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.113304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.123842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:39.129387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:39.130321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:39.131600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:39.135368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691d0d-9675-4bd2-8e3c-968cbc97c964 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.135760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 319/627] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:39.136602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.154002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.180941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.181886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.182307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.182767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.183724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:39.184167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.185989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.205638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 309/628] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:39.206661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.244172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4385f-278a-44a4-ac4a-b615050e1448 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.278137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.278777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.279049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.279312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.286187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.287652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:39.299511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:39.300605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:39.301736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:39.302368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 320/629] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:39.303307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.328815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.333971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a516b21-6eb3-45f4-a592-53a4e6e90b6f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.385070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.386004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.386425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.386867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.387724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:39.388163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.390215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.401921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 310/630] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:39.403035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.419993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f6016-d7f1-47ed-aec2-52f17bf3def9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.442506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.456125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.456750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.457047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.457312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.529846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d6b2e-9d9f-465e-b13a-41580b4104a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.530065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.531502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.550629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.572059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.572994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.573428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.573842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.575166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:39.575631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.577973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.599452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 311/631] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:39.599452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.648910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 321/632] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:39.649594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.708239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.717957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79316bd-80c2-4f68-bf07-2ac2f4540183 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.758203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3f94-0bc5-4dd1-81a2-a3633bbcc6fe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:39.778833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:39.798482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.799168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.799439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.799701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/e64871a43e564c9bbf7ebfdbd512a1f3/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.816893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=e64871a43e564c9bbf7ebfdbd512a1f3, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.818374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.840795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 322/633] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:39 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/e64871a43e564c9bbf7ebfdbd512a1f3/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:39.840795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.877025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:39.877616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.877894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.878140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.891874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:39.893485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-51c19b9e-e33e-483f-86e1-1842fd9f884f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:39.911519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:39.912963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:39.913861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:39.913861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:39.913861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:39.913861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:39.917875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 312/634] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:39.919322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217043b-3991-48ff-b5c5-7f0c02352cef None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:39.919600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217043b-3991-48ff-b5c5-7f0c02352cef None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:39.919874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217043b-3991-48ff-b5c5-7f0c02352cef None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:39.925962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 313/635] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:39.926918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217043b-3991-48ff-b5c5-7f0c02352cef None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:39.975033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47774c6-a78a-48d8-a593-e883e7ae8978 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.020245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.021037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.021325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.021592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.034885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.036347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.046009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217043b-3991-48ff-b5c5-7f0c02352cef None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d323f41f-6d44-45f3-b301-0ee89ba77384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:40.051879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:40.052789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 323/636] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.057871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.073200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:40.074380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:40.088747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.089211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.089479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.089799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.090123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.090564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:40.090720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.092089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5dddd-cb70-4c43-86d2-3d54cf0b2ef4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.105636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 314/637] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:40.107217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680166-9415-4f96-a390-4ebd8f5e681c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.107717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680166-9415-4f96-a390-4ebd8f5e681c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.108075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680166-9415-4f96-a390-4ebd8f5e681c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.109724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 315/638] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:40.114386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680166-9415-4f96-a390-4ebd8f5e681c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.118923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.160979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 324/639] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:40.160979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.232373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680166-9415-4f96-a390-4ebd8f5e681c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.268219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.268860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.269132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.269389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.270068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:40.270360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.271718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ac7be-aa96-47fb-86ed-7086bd1cd7d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.280749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb71d1d-d84a-47a3-ac5f-4c5c45bf217b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.283825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 316/640] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:40.284536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.284965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.285203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.304141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:40.306987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:40.325397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.326076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.326349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.326611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.334506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.335968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:40.347906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:40.349072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 325/641] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:40.354976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.366321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.375701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.385475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 317/642] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:40.386485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.478282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b8881-8f62-40a0-8571-992b7bf42d26 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.490637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae8e04-a7f8-489f-8fa0-be2b13134514 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.513425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.514062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.514366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.514625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.526200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:40.527233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.527857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.528125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.528384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:40.531795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:40.532798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 326/643] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:40.538724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.539508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.539508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:40.552196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:40.553147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:40.554181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 318/644] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:40.555310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.610978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.613583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.643502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf5577-aefe-4d72-8750-21085b96d0f2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.660425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3246f-30af-4054-90c9-02fe2e7b4eb3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.679008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.679626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.679899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.680169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.681398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:40.681701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.683167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.694761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.695570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.696008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.696411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.700114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 327/645] 10.222.0.49 () {56 vars in 1137 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:40.700948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.709763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:40.715403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:40.716478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:40.717524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:40.719511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 319/646] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:40.720412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.811409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ac20-ec18-4d01-9954-100ff8f49de1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.825119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28483f0-8fad-43dd-b60d-2623ff7aa556 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.839128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.840261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:40.849568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.851060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.851640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.852203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.853348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:40.854232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.856307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.861686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:40.862297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:40.862946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:40.863237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:40.863979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:40.864274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:40.865716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:40.867995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 328/647] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:40.868940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.887777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 320/648] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:40.887777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:40.981533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797a4d9-a2ce-4e26-9d2a-e7427b49b45e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:40.992270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc672d1-e1a1-4833-a645-66bb951c1b40 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.019257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.019883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.020154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.020418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.028301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.029670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.029945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.030211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.030959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:41.031266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.033276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.042119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 321/649] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:41.043598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.082513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.082895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.095026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.096815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.150023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09b980-9cc9-4542-ac3a-f3010c199354 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.183474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.184111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.184385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.184652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.194804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.196227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.197884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 329/650] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:41.198653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:41.215283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:41.216237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:41.221899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:41.221899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:41.221899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:41.221899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:41.221899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 322/651] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:41.221899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.304684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.306743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.316044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16478-1b9d-47c6-ad95-d83af35e9d2f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.320657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ff514-35e7-415c-b76f-81dcce97dc91 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.354498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.355046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.355330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.355599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.356397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=9958737194be4488a1e09e6e48557d75) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:41.356703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.358173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.358519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.359441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.359720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.359991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.365660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 330/652] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/role_assignments?user.id=9958737194be4488a1e09e6e48557d75&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:41.366713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.371674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.373184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:41.394253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:41.395249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:41.396232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:41.396232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:41.396232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:41.396232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:41.404868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 323/653] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:41.406036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.495982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f3fb-7751-4656-877b-e3494e1fc514 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.538987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.540103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.540586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.546998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b966665-aab6-4a71-9076-cbc122a3dd5e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.547242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.562813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.564402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:41.579223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:41.580566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:41.581933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:41.581933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:41.581933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:41.581933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:41.581933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 331/654] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:41.581933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.593961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.600887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.601351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.601689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.602021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.602968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:41.603609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.606082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.616301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.620796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 324/655] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:41.626123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.703684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7211ad-acd8-4ace-9611-0e3b2dc80f02 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.748480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b8ce6-c751-4122-a347-ce4df1299006 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.754223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.755069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.755531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.755952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.757189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:41.757644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.759184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.781317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 332/656] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:41.782991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.790793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897edc-9458-4336-babc-3c9806762476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.791495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897edc-9458-4336-babc-3c9806762476 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.791824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897edc-9458-4336-babc-3c9806762476 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.792143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897edc-9458-4336-babc-3c9806762476 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.792880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56897edc-9458-4336-babc-3c9806762476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:41.793184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56897edc-9458-4336-babc-3c9806762476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.794560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56897edc-9458-4336-babc-3c9806762476 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.807192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 325/657] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:41.808164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897edc-9458-4336-babc-3c9806762476 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.860733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.876053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:41.902006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41da39-59e8-460b-928b-de1248862d6e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.920788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897edc-9458-4336-babc-3c9806762476 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:41.938960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.939606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.939877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.940142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.947898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.949528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.955174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:41.955876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.956172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.956409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:41.961760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:41.962888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 18:24:41.964648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:41.965332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 333/658] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:41.965665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:41.967578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:41.969681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0bab93d4-2e2a-4013-a7db-515da78a2e12 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:41.984648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:41.985815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:41.986801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 326/659] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:41.987402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a028dc-ba73-490c-ba08-f657691e22bd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:41.987402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a028dc-ba73-490c-ba08-f657691e22bd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:41.987664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a028dc-ba73-490c-ba08-f657691e22bd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:41.990779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 327/660] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:41.992111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a028dc-ba73-490c-ba08-f657691e22bd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.075913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec03573-bf92-48ad-817a-598597bf78ef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.098438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a028dc-ba73-490c-ba08-f657691e22bd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.102762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.113788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.114351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.114619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.114885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.119126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.122974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.124517 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.136301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:42.136925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:42.137830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:42.138800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:42.139245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.139245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.139245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.139442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:42.139999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.141339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01127f4a-a012-43a0-b386-32118e8da1ab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.142988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 334/661] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:42.143861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.171238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 328/662] 10.222.0.49 () {56 vars in 1139 bytes} [Sun Jul 31 18:24:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:42.171238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.171238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.171238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.186238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:42.187555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:42.257474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b806194-fbbc-42aa-b96f-c17793ceac6f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.288814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 329/663] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:42.288814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.294372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.294997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.295270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.295537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.296324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:42.296752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.298213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.316687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 335/664] 10.222.0.49 () {56 vars in 1137 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:42.317543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.368401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.398510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96eaa9-da6c-4bba-a493-7cae447ec016 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.407586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.433012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03860f96-bb4d-4685-9ff4-432218b3fb26 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.433421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.434042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.434311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.434583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.435975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:42.435975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.437064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.453477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 330/665] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:42.453477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.478485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.479110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.479392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.479653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.554536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.556013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.557048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b5d6a-e8cc-481c-b633-ce435d7a4678 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.592298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.592974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.593259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.593558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.600952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.604924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:42.611170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:42.612205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:42.613110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 331/666] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:42.613848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.644383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.670451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 336/667] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:42.670451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.724983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5d985-fa46-42e3-91d6-412bd6e81611 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.760180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.761359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.761803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.762077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.768970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c6360-46aa-4788-918f-1c614a58622b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.803043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.803660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.803940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.804198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.805010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=20341bc6269f477c952c25437ab7a2d1) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:42.805313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.806781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.820803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 337/668] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/role_assignments?user.id=20341bc6269f477c952c25437ab7a2d1&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:42.820803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.824874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.846098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.848381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.888978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:42.922642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbfc0a-27b7-46a6-a4c9-ee7bcdb3b3b3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:42.959008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:42.959991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:42.960463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:42.960969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:42.961804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:42.962227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:42.963714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:42.978023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 338/669] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:42.978023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:42.988098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 332/670] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:42.989276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.070097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.106154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f180860-eabe-4b32-aa95-270d2ca541ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.119464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e048-e531-4ef5-b1c0-ca9621b59bcf None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.148036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.153715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.154372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.154648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.154916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.161199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.162128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.162599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.163585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.166913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.168382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.179073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.180450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:43.186885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:43.187813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:43.188708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:43.188708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:43.188708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:43.188708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:43.190252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 339/671] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:43.195355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:43.203738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:43.204774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:43.205674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:43.205674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:43.205674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:43.205674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:43.208478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 333/672] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:43.209454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.314374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261babe7-3ce9-4264-b59a-4433fe236558 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.343382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b2e9-e4f6-420c-bc81-10af2876dbe1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.344377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.361209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.362582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.363071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.363553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.378233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.380399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.380593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.381652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.381937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.382210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.382895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:43.383201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.384555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.402312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 334/673] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:43.402312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:43.403036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:43.403879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:43.404813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:43.404813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:43.404813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:43.404813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:43.405530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 340/674] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:43.406804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.453350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.505864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46bf51-117e-4850-8f28-9a0d6fc64093 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.532530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7c41-61f0-4169-bbd0-877fa3994f7d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.542944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.543901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.544233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.544509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.545528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:43.545968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.547364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.558699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 335/675] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:43.559583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.574826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.575925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.576200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.576463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.583763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.585553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6eedfbd8-a96a-4e1b-84c8-a5d18acbf741 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:43.597746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:43.598620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:43.599500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 341/676] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:43.600421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc895f-4f69-4122-830a-40a2ff40464e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.600524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc895f-4f69-4122-830a-40a2ff40464e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.600708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc895f-4f69-4122-830a-40a2ff40464e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.605020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 342/677] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:43.606572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc895f-4f69-4122-830a-40a2ff40464e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.644647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.662695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.666052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b975b-d136-4c35-b1ee-428ac4df69ad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.700312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.701216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.702089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.702505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.703416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:43.703898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.705468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.714811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc895f-4f69-4122-830a-40a2ff40464e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.722195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 336/678] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:43.723208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.753851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.754483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.754762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.755038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.762844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.764274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8678d286-255c-4c3c-ac3c-b383097b0803 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:43.775551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:43.776496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:43.777498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 343/679] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:43.778104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.778104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.778776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.795658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:43.797010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:43.833588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c4cc9-726a-48ca-ac94-d8368ff54dfe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:43.866738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 344/680] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:43.867586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.880826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:43.882056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:43.882530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:43.883014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:43.883977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:43.884604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:43.886241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:43.899232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 337/681] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:43.900374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:43.903379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.923373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:43.986000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d5acb-7f76-451e-a3cc-b4b8f6fcaa5f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.012575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902f06-3e2f-475f-8f31-d52135801286 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.020550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.021237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.021507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.021767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.028925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.030399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:44.043006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:44.043997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:44.044947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 345/682] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:44.045426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.057898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.058473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.058759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.059005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.059801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:44.060268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.061535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.080133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 338/683] 10.222.0.49 () {56 vars in 1135 bytes} [Sun Jul 31 18:24:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:44.081052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.142503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:44.158469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81370490-9a74-45bd-a140-f4812d5e35ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.164578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:44.190008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921132f9-2326-4f52-80da-0a749a2a68e3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.235443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.236148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.236338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.236629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.237473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=f590f668ad724bf087b164bcf3f543ba) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:44.237780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.239231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.256939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 339/684] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/role_assignments?user.id=f590f668ad724bf087b164bcf3f543ba&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:44.256939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.328067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.328881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.329155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.329422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.344952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:44.351879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:44.353098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:44.354066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:44.357038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 346/685] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:44.357859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.393728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564a4d8-80d9-484f-9ce9-edf24dd8910a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.447342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.447976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.448247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.448511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.449297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:44.449592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.451020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.467534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 340/686] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:44.468788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.475471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:44.478262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ccf2a-5aa9-41e7-a761-57fa6bd6d496 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.520868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.521416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.521691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.521956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.540561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:44.585184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ae0d-627c-4635-b42c-c953921d5660 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.609445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.611129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.625488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.626554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.627022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.627480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.628408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:44.629053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.630720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.640918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 341/687] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:44.642071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.721618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:44.734935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 347/688] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:44.734935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.778776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71f0ff-9cbc-4d35-a51b-cad45fe5f723 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.791001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:44.826184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.827422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.828007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.828720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.830201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=7e84fa2255614cde84495eedcc0971a9) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:44.830972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.833199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.848835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 342/689] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/role_assignments?user.id=7e84fa2255614cde84495eedcc0971a9&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:44.848835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:44.866393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97dd72-0713-484f-bbea-36c249735cdf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:44.934672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:44.935600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.936021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.936465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.953531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:44.955040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2173f47f-aba7-4597-a426-2352ef2d3425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:44.977062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:44.978053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:44.978917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:44.978917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:44.978917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:44.978917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:44.981631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 348/690] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:44.984082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88aab5-35a5-444f-80bc-fb8a0b8223c0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:44.984672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88aab5-35a5-444f-80bc-fb8a0b8223c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:44.985238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88aab5-35a5-444f-80bc-fb8a0b8223c0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:44.987273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 349/691] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:44.988194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88aab5-35a5-444f-80bc-fb8a0b8223c0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.010236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58b38-9d54-45f5-ad23-a24284ec9d14 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.040547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.054011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.054683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.054954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.055501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.071556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.073560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-753ddd7c-c952-4f29-9468-6eca776f87c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:45.090242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:45.091241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:45.092124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:45.092124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:45.092124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:45.092124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:45.092124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 343/692] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:45.092869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.093683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.093997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.112685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:45.113935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:45.125876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.135923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88aab5-35a5-444f-80bc-fb8a0b8223c0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.179360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.179989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.180402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.180673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.181411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:45.181713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.183089 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.195180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 350/693] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:45.196082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.197540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 344/694] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:45.205416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.334245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37514aa-e001-4975-a68f-f58625ffaf00 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.346116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.353387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08ca8-27a4-449a-91d1-a6f7300d71fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.372001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.372484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.372774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.373048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.381650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.383114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:45.395925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:45.397116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 345/695] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:45.398118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.399543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.400423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.401151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.401606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.403523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:45.403829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.405246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.421129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 351/696] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:45.422064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.445301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.515094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ba286-a843-47eb-bbf7-131fe7abe3d1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.550180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2f774-d672-4579-8e86-b4f43df1933b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.550490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.551556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.552177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.552843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.553803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.555614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:45.571728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:45.572743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:45.574488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 346/697] 10.222.0.49 () {56 vars in 1129 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:45.575335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.597913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.598522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.598795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.599546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.600590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:45.601049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.603178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.613205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.628299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 352/698] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:45.629382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.683693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a110aa6-db8d-4f3b-a945-c774240e0f58 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.718663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.719502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.719870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.720236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.727188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.729533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:45.741813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:45.743500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:45.745586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:45.750144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 347/699] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:45.750144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.758484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab8c8-b93c-4ef6-afea-dd7684badbcd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.796024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.796751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.797058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.797369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.798088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:45.798388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.799743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.804663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 353/700] 10.222.0.49 () {56 vars in 1124 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:45.805550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:45.858329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:45.866875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5a881-d0b3-434e-a3f7-ca111fe3fde9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.902187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.902968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.903334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.903720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.935935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04ecbc-f0fb-48bd-a62b-30360491b672 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:45.974763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.976348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.981478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:45.982106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:45.982378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:45.982645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:45.983398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:45.983692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:45.985197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:45.994651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.002279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 354/701] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:46.003201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.083290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.092668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 348/702] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 18:24:46.093719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.135144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd386-22bf-40fa-8a0f-05bd50281c62 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.169635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.170424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.170788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.171144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.172022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:46.172409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.173954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.179990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 355/703] 10.222.0.49 () {56 vars in 1124 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:46.180917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.282180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.292536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f08b3b-b5ec-4920-b131-775c97118a41 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.331640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b809f-dae5-4fd4-b4bd-b282caaf158d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.377925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.379572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.379844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.380107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.384409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.385792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.385792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.385792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.390932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=e64871a43e564c9bbf7ebfdbd512a1f3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:46.390932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.390932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46449638-3e22-461b-b93a-747c957cdd7c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.394999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.395286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 349/704] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/role_assignments?user.id=e64871a43e564c9bbf7ebfdbd512a1f3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:46.396510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8497c7-7e5d-4e05-ad07-7808d1229341 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.397224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.404878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8497c7-7e5d-4e05-ad07-7808d1229341 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.405349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8497c7-7e5d-4e05-ad07-7808d1229341 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.406951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 350/705] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:46.413979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.414252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.414690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:46.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:46.417229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:46.418134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:46.418134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:46.418134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:46.418134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:46.424520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 356/706] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:46.426640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.444158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:46.445835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:46.542459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 351/707] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:46.543543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.578480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7efba-5aae-4dff-b31b-49ed9f382527 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.585203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.593433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.614827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.615513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.615787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.616055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.616749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:46.617076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.618461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.633964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 357/708] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:46.639065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.709611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabbc7-e042-486f-9a88-6ee7b0dfa5af None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.751055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c7a2a-8fe1-43cf-8375-0041b1bbf109 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.757695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.758462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.758740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.759009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.760858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.762340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:46.774615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:46.775602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:46.776453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:46.777582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 352/709] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:46.782367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.791043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.791694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.791970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.792239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.801248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.803365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:46.820112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:46.821387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:46.822913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 358/710] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:46.827512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.862649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.874310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:46.911879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57ffd-e92b-4d8f-adcc-6cc4bf1670dc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:46.961571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:46.962221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:46.962500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:46.962775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:46.963470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] RBAC: Adding query filter params (name=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:46.963762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:46.965168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:46.977141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 353/711] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:46.978097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:46.986595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4f3a9-c539-41ab-bdb8-964a8dfd3de2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:47.021521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:47.022527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.022991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.023463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.024395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:47.024922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:47.026553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:47.044260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 359/712] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:47.045797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:47.083472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfabf06-3838-44d1-b5e4-845b78604b9d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:47.132062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:47.153836 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:47.160723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:47.161389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.161665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.161989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.162623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] RBAC: Adding query filter params (type=network) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:47.162925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:47.163306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c9b9a-b8d9-4604-a9cc-e6d2ba7fc3e1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:47.164293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550086a-b54f-417b-a5c7-40e8f199b8dc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:47.170648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 354/713] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:24:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:47.172062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29a43-1216-45f7-8d49-8e4001fd93f5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.172279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29a43-1216-45f7-8d49-8e4001fd93f5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.172551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29a43-1216-45f7-8d49-8e4001fd93f5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.178315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 355/714] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:47.180001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.180283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.180558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.203064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:47.204419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.204919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.205395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.206390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=9958737194be4488a1e09e6e48557d75) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:47.206902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:47.208542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:47.212821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:47.214146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:47.220905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 360/715] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:47 2022] GET /identity/v3/role_assignments?user.id=9958737194be4488a1e09e6e48557d75&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:47.344705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 356/716] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 18:24:47.375255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:47.486665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37dbb3-95ac-40ae-a126-99884ee95ca9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:47.523173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:47.523825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.524101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.524370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.535832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:47.537370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:47.552232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:47.553199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:47.554521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:47.554521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:47.554521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:47.554521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:47.554521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 361/717] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:47.559914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:47.689113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2c041-e667-4d52-823e-191b2f144320 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:47.705738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:47.734841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:47.735464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.735740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.736014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.736751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:47.737025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:47.738409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0c789-9581-4c5b-a922-6112086baaa1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:47.753383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 357/718] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:47.759508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:47.776271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:47.874328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f782f-3fbc-4a59-bad3-b8b5d4bd9a7b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:47.926569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a434-305d-496c-ac63-0f86a04d7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:47.927200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a434-305d-496c-ac63-0f86a04d7a8a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:47.927477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a434-305d-496c-ac63-0f86a04d7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:47.927746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a434-305d-496c-ac63-0f86a04d7a8a None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:47.928442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a434-305d-496c-ac63-0f86a04d7a8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:47.929898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a434-305d-496c-ac63-0f86a04d7a8a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:47.963568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 362/719] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:24:47 2022] POST /identity/v3/users => generated 254 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:48.565988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ece1a8-8f6a-4637-85c4-7174731a3982 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:48.566499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ece1a8-8f6a-4637-85c4-7174731a3982 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:48.566499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ece1a8-8f6a-4637-85c4-7174731a3982 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:48.568067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 358/720] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:48.575241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e635c97-2f26-4f74-a812-59b6985b7512 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:48.575672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e635c97-2f26-4f74-a812-59b6985b7512 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:48.575845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e635c97-2f26-4f74-a812-59b6985b7512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:48.596581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2e635c97-2f26-4f74-a812-59b6985b7512 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:48.597863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e635c97-2f26-4f74-a812-59b6985b7512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:48.664548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:48.680168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 363/721] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:48.720000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ece1a8-8f6a-4637-85c4-7174731a3982 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:48.841269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ece1a8-8f6a-4637-85c4-7174731a3982 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:48.877571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00483a29-8dc4-4008-a3e5-30b9adc88ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:48.878191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00483a29-8dc4-4008-a3e5-30b9adc88ebe None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:48.878467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00483a29-8dc4-4008-a3e5-30b9adc88ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:48.878739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00483a29-8dc4-4008-a3e5-30b9adc88ebe None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:48.879428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00483a29-8dc4-4008-a3e5-30b9adc88ebe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:48.880809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00483a29-8dc4-4008-a3e5-30b9adc88ebe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:48.916343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 359/722] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:24:48 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:49.207839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b209f5-4d66-454e-8539-5441f7892eaf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.208220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b209f5-4d66-454e-8539-5441f7892eaf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.208393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b209f5-4d66-454e-8539-5441f7892eaf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.210063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 364/723] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:49.215220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.215722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.227109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.254142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:49.255659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:49.284346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:49.355003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 360/724] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:49.390368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b209f5-4d66-454e-8539-5441f7892eaf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:49.498516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b209f5-4d66-454e-8539-5441f7892eaf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:49.538919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8cd98-1b4b-4637-9b73-00d9248ed46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:49.539571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8cd98-1b4b-4637-9b73-00d9248ed46c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.539836 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8cd98-1b4b-4637-9b73-00d9248ed46c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.540108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8cd98-1b4b-4637-9b73-00d9248ed46c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.557849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8cd98-1b4b-4637-9b73-00d9248ed46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c2b8cd98-1b4b-4637-9b73-00d9248ed46c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:49.562921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:49.563840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:49.564665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 365/725] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:49.571908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:49.595284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccc728-4a99-46bf-8629-f993f0bba866 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.596129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccc728-4a99-46bf-8629-f993f0bba866 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.596920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccc728-4a99-46bf-8629-f993f0bba866 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.599009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 366/726] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:49.604929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.605365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.606012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.626511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:49.628466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:49.687439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:49.708226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 367/727] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 18:24:49.734183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:49.777395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0c2f5-e914-4015-8616-13cd2bb2c03b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:49.814755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:49.815759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.816174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.816575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.817499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:49.817969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:49.819803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecda80-8b7f-4653-95ae-d5f48a5eb98d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:49.829474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 361/728] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:49.831057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d57c4b-f7c7-4cb2-8404-017728e42fdc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.831531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d57c4b-f7c7-4cb2-8404-017728e42fdc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.832105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d57c4b-f7c7-4cb2-8404-017728e42fdc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.834219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 362/729] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:49.835157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d57c4b-f7c7-4cb2-8404-017728e42fdc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:49.855182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93097f-1aab-452f-9374-ed826925b86b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:49.873130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:49.898464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab700ee-3aee-41a7-b884-1f20a9bac213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:49.899452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab700ee-3aee-41a7-b884-1f20a9bac213 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.899738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab700ee-3aee-41a7-b884-1f20a9bac213 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.900005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab700ee-3aee-41a7-b884-1f20a9bac213 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.901054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab700ee-3aee-41a7-b884-1f20a9bac213 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:49.902682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab700ee-3aee-41a7-b884-1f20a9bac213 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:49.928868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 368/730] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:24:49 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:24:49.930875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:49.931441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:49.931902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:49.971780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:49.974940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:49.999077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d57c4b-f7c7-4cb2-8404-017728e42fdc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.033541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:50.064854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 369/731] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:50.081109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.082259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.082791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.083249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.096603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.098467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.102807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:50.119463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:50.120370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:50.121845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:50.121845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:50.121845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:50.121845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:50.121845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 363/732] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:50.126330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.209398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:50.218835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85ea61-e813-48bb-8f58-2bf69214e194 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.240945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e81af5-8580-4569-8b0b-f31ffd56cc66 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.257098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.257751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.258029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.258300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.270770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:50.275807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:50.276711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:50.277591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:50.278215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.278215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.278215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.278215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.279142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:50.279516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.281145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.285801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 370/733] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:50.293446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.295144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 364/734] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:50.300514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.369888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:50.408415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8836c98-b746-4cce-b816-d935a044ab71 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.442999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.443647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.443925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.444195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.452618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.454198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.457086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:50.478091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:50.479100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:50.480024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 365/735] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:50.485657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.498768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ba84-b2aa-4e03-b717-6fb4ccfe28ac None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.552983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.553536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.553812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.554085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.554808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:50.555111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.556734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.576881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 371/736] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:50.581562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.689247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26720a9-ff62-4326-93f2-ca2f0bc46491 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.702727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:50.708784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cd814-dadc-4baf-8a00-15024acee6f6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.741472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.742123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.742400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.742661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.753903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.755379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:50.770757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:50.771658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:50.772546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:50.772546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:50.772546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:50.772546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:50.772546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 372/737] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:50.780175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.788032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.788986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.789476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.789932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.790912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:50.791401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.793036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6323-a866-4222-8d3a-20efc22aab9e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.813296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 366/738] 10.222.0.49 () {56 vars in 1135 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:50.816602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8bd3d-57fa-4e88-8f75-87f2956cb6a9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.817260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8bd3d-57fa-4e88-8f75-87f2956cb6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.817720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8bd3d-57fa-4e88-8f75-87f2956cb6a9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.819411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 367/739] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:50.820520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8bd3d-57fa-4e88-8f75-87f2956cb6a9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:50.869664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:50.910066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b869c5-7471-4f5f-8ee1-b8dabf73fab6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.934132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8bd3d-57fa-4e88-8f75-87f2956cb6a9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:50.951591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.952564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.952876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.953146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.954287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:50.954590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:50.956240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f30c3-c45a-4936-8162-15ad146a9ae1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:50.977351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 373/740] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:50.977351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.977351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.977351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.985745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:50.986590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:50.987046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:50.987535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:50.996750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:50.998538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:51.004031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.075444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.077238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.080798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 374/741] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:51.081777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.129103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.188809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 368/742] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:24:51.189932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.221501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185ec0b-420c-4a09-9aa2-04c2801903f4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.259922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.260810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.261278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.261611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.272376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.273968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.277081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:51.289704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:51.290823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:51.291752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:51.300405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 375/743] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:51.300405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.313373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a0885-2c2b-42e5-bbfc-7223f53f3dc8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.351484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.352042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.352315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.352580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.363578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:51.368586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:51.369661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:51.370499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:51.374854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 369/744] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:51.375826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.382355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.431422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0319a-e5ce-4e8d-b7c9-b3d0022aa1ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.466509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.466899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.467155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.467416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.478927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.480403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.489804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203ba3-8935-490c-8fda-91a226ff3a46 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:51.494990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:51.496004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:51.497113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:51.497113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:51.497113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:51.497113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:51.498866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 376/745] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:51.499899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.531325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.531954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.532209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.532474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.533305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:51.533595 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.535010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222809-e1bb-4274-9a70-edf380dc0413 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 370/746] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:51.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddbf8f-2b4f-453a-8631-468684e66746 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddbf8f-2b4f-453a-8631-468684e66746 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddbf8f-2b4f-453a-8631-468684e66746 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 371/747] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:51.557117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddbf8f-2b4f-453a-8631-468684e66746 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.565274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.614676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57acd3a-9915-45c0-8b08-48f4e3d068d0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.623008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.656142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.656928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.657201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.657466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.658493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:51.658793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.660579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.664432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddbf8f-2b4f-453a-8631-468684e66746 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.675986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 377/748] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:51.676687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.698684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.699355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.699613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.699878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.700584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:51.700894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.702242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb7b53-223f-4397-a381-2576509672ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.714310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 372/749] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:51.715367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b8529-6695-433b-aef5-79ac2472462b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.715648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b8529-6695-433b-aef5-79ac2472462b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.715908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b8529-6695-433b-aef5-79ac2472462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.734432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-322b8529-6695-433b-aef5-79ac2472462b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:51.735600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-322b8529-6695-433b-aef5-79ac2472462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:51.787328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ff521-822e-4938-b349-ac260c6e9d26 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.800794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.810415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 373/750] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:51.811607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b8529-6695-433b-aef5-79ac2472462b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.829708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.830488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.830853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.831263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.843068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.844573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:51.860382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:51.861507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:51.867178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:51.867178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:51.867178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:51.867178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:51.867178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 378/751] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:51.867178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:51.871581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:51.919927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b8529-6695-433b-aef5-79ac2472462b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.957129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:51.958077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:51.958348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:51.958621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:51.969273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:51.970870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:51.979296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf265-66c3-4da8-9702-35ad39cb980a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:51.987512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:51.988393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:51.994526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:51.994526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:51.994526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:51.994526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:51.994526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 374/752] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:51.994526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.014335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.015039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.015355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.015622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.022430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.023889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:52.035139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:52.036011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:52.036948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 379/753] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:52.038197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.066784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.099743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25bc08-bcfe-4225-af30-066c80b333c4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.102253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.135919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.136837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.137104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.137382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.138214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.139343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.143963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b4bd5-3c78-435b-b60b-ee7919fc2f88 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:52.148653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:52.149599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:52.151092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 375/754] 10.222.0.49 () {56 vars in 1129 bytes} [Sun Jul 31 18:24:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:52.151937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.185798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.186513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.186791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.187091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.187769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.188073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.189461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.207505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 380/755] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.208737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.257629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dd1c6-2980-4d2f-80d2-356616248ff5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.293525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.294148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.294418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.294709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.295391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.295686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.297134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.308198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 376/756] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.309020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.309847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.325532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230e19e-a5f3-411a-b823-1e916d4b9b78 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.338538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.361281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.362038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.362367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.362636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.363384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.363680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.365131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.376412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 381/757] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.377232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.409297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f10807-be39-4014-9df2-6d89888b277c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.442686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.443317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.443589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.443851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.444536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.444857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.446209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830420b-6293-4b25-9446-c9846b5989bb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.450456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 377/758] 10.222.0.49 () {56 vars in 1124 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.451575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce83f37-9ad3-4b2e-9570-35ff7772c0df None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.451851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce83f37-9ad3-4b2e-9570-35ff7772c0df None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.452118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce83f37-9ad3-4b2e-9570-35ff7772c0df None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.453518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 378/759] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:52.457661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce83f37-9ad3-4b2e-9570-35ff7772c0df None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.477415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a944e0-8053-4357-b097-083172d2ddde None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.509812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.510449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.510723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.510987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.517732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.519176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:52.530340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:52.531383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 382/760] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:52.532305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.544385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.554790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.565617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce83f37-9ad3-4b2e-9570-35ff7772c0df None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.598913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.599547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.599834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.600097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.607718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.609212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:52.620703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:52.621698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 379/761] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:52.622628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.639775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aada4f5-4e65-4ec5-9822-817ba57dcfdc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.674993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.676002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.676489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.676954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.677749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=e64871a43e564c9bbf7ebfdbd512a1f3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.678058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.679488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8cfa6-93ea-497d-8796-aa9bc42a2b09 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.685219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 383/762] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/role_assignments?user.id=e64871a43e564c9bbf7ebfdbd512a1f3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.686422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.686701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.687133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.706583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:52.707806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:52.740868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27c130-2460-448b-aefe-178839430d36 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.778450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.778723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.779327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.779669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.779909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.780643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.780972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.782369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.784095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 384/763] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:52.785002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.801708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 380/764] 10.222.0.49 () {56 vars in 1124 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.801930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.821831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:52.900062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f7084-da8f-456e-8dfb-1d5de421d002 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.910975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc983a8-e1dc-4971-b47c-c2225dadfc6c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:52.933887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.934440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.934715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.935042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.935731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.936037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.937544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.947449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:52.948302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:52.948750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:52.949213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:52.949829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 385/765] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.950405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:52.950622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:52.951153 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:52.952891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:52.971882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 381/766] 10.222.0.49 () {56 vars in 1149 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:52.973178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.023329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.056087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.070638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478fce-ec22-4e45-8e8e-78c2282f3389 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.088975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d51c5-2b15-414d-a3a5-820b08b2ec36 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.105565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.106203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.106461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.106722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.107465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] RBAC: Adding query filter params (interface=public, service_id=b16e67d35e174fa4b7fba8b29a51eed9, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:53.107793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.109270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.121649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 386/767] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/endpoints?service_id=b16e67d35e174fa4b7fba8b29a51eed9&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:53.122493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.123866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.124604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.124912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.125184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] PATH_INFO: `/v3/services/network` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.125820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.127189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:53.138292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:53.139301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:53.140178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 382/768] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:52 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:53.140996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.241084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854af431-7377-403a-8075-0c16bcbf958d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.255378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055acd88-2baf-4e70-8225-cfc36823cdbd None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.266013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.277575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.278377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.278666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.278962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/9958737194be4488a1e09e6e48557d75/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.280993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=9958737194be4488a1e09e6e48557d75, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.282437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.290893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.291523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.291793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.292070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.304800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 387/769] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:53 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/9958737194be4488a1e09e6e48557d75/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:53.304800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.305104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.378143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.379617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.423639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505adcdc-0364-4738-b589-1f80a29d5b64 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.465469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.466110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.466386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.466655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.467315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.468830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.475225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 388/770] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:53.476202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.519119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 383/771] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:24:53.526062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.547278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.584658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.629122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5346d8-7c7b-48a7-9cf4-cc3b88f73dd2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.656243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d937-67a2-48ea-a46f-dc70b0ada82d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.677052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.678000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.678920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.679400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.680509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:53.681249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.683517 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.690852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 389/772] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:53.694891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.695630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.696205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.696575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.697135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.709032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.710664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:53.726389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:53.727515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:53.728418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:53.728418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:53.728418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:53.728418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:53.729270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 384/773] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:53.735892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.830091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2812032-ab8b-48e1-89f7-e7c0c4a7523f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.844338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.850123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:53.864790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cdb0f-1d8a-4692-ac1b-38bf99683ebe None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:53.868996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.869753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.870105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.870440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.871189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:53.871553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.873031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.884316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 390/774] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:53.885660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:53.925003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:53.926038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:53.926511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:53.926971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:53.927858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:53.929424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:53.931005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:53.950836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 385/775] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:53.955977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:54.026817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325967-0619-4714-849a-c2368f2acdc7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:54.093261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:54.093811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.094084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.094348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.095097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] RBAC: Adding query filter params (interface=public, service_id=68180508f85e449bbdb541ec833112dd, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:54.095397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:54.096789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:54.109621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 391/776] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/endpoints?service_id=68180508f85e449bbdb541ec833112dd&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:54.115313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:54.121271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ca9ab-6e90-40c4-a434-6f51e02c4e1f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:54.170402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:54.198430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:54.199165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.199417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.199689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.218013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:54.219744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:54.234196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1bb76-257f-4ed0-ba2c-56a4888740ca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:54.250515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:54.251476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:54.252351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 386/777] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:54.265837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:54.273660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:54.274360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.274699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.275036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.275746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:54.277237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:54.283252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 392/778] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:24:54 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:54.288827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:54.440893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3ff5f-8b8c-49fc-9e56-688f51fb532c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:54.480020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:54.480679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.480992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.481428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.482180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:54.482476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:54.484034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc9578-7e70-4ea7-819e-ec51f226c6ac None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:54.496943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 387/779] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:54.503827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:54.624655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2be632-db8a-4f3f-98b2-341599da7def None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:54.629700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:54.678414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:54.679051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.679328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.679597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.680401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=4242cff2780f478099110d0586ad67b3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:54.680696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:54.682165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42915580-a525-4fe1-902a-e158fc7444da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:54.693745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 393/780] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:54 2022] GET /identity/v3/role_assignments?user.id=4242cff2780f478099110d0586ad67b3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:54.923230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70384f52-a6b0-40fb-89fc-18c4b39684fd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.923766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70384f52-a6b0-40fb-89fc-18c4b39684fd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.923766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70384f52-a6b0-40fb-89fc-18c4b39684fd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.925128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 388/781] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:54.930390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:54.930669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:54.930938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:54.953841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:54.955015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:55.039615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 394/782] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:24:55.073636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70384f52-a6b0-40fb-89fc-18c4b39684fd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:55.170631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:55.211637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70384f52-a6b0-40fb-89fc-18c4b39684fd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:55.256498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:55.257561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:55.258036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:55.258501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:55.277657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:55.283393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:55.284305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:55.285184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:55.287827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 389/783] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:55.295498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:55.409772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:55.414637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65adea-6e0a-4ddd-9254-4c84b51dd723 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:55.450227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:55.451245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:55.451715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:55.452180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:55.453182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:55.453732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:55.455294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:55.467876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 395/784] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:55.473668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:55.589383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577926-7842-4268-a1e7-808234d7d43f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:55.628109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:55.639317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:55.640002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:55.640500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:55.640988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:55.655607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:55.657090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e5903d39-d47a-4de7-b7c7-ca956513a8ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:55.671628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:55.672530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:55.673467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:55.673467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:55.673467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:55.673467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:55.673467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 390/785] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:55.678403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:55.680581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f68cb-52b6-43fc-9cf4-b0d239abdcff None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:55.680882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f68cb-52b6-43fc-9cf4-b0d239abdcff None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:55.681150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f68cb-52b6-43fc-9cf4-b0d239abdcff None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:55.682672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 391/786] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:55.690547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:55.690811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:55.691079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:55.723482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:55.724587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:55.838608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbae34-432f-47c6-a4cb-3d76dc0aca96 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:55.847877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 392/787] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:55.887909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:55.892874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:55.893519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:55.893800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:55.894069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:55.894775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:55.895096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:55.896481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:55.917724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 396/788] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:55.923634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:55.947497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.069713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f585a4-f738-464c-8c99-1b21fa3d0b67 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.084538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08873bf4-a03b-4092-8c52-54da57364d94 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.099949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.127003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.131469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.131469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.131469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.139578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.140559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.147794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.148462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.148732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.149045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:56.167308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:56.169544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:56.171687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 397/789] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:56.176787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:56.177657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:56.179087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:56.179968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:24:56.180813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:56.183363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 393/790] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:56.193789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.267669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.283818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77666c63-0078-49dd-ab84-e45ac4fa2943 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.335533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.336084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.336363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.336917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.337733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:56.338042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.340012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce735d9-f7a0-4735-b902-486404cf8843 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.361682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342be20-bc05-410f-815f-b8d71d6c93e4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.362374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 398/791] 10.222.0.49 () {56 vars in 1135 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:56.363821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff229e1-7be2-4e4b-a96f-85a565ecd71e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.364116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff229e1-7be2-4e4b-a96f-85a565ecd71e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.364382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff229e1-7be2-4e4b-a96f-85a565ecd71e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.370964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 399/792] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:56.371757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff229e1-7be2-4e4b-a96f-85a565ecd71e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.404074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.405230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.406423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.407329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.408802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:56.409633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.411506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a52a2d-ab31-4be9-aed5-32d4f324a4c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.422529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 394/793] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:56.424892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.425350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.425758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.448746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:56.450157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:56.490165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.493797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff229e1-7be2-4e4b-a96f-85a565ecd71e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.537132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.537792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.538070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.538341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.542390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 395/794] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:56.543458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.550370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.561229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.565994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:56.584323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:56.585411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:56.586476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:56.586476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:56.586476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:56.586476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:56.588206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 400/795] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:56.589126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.668023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a5e32-7e6f-4824-af2c-22c664549f75 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.704809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.705525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.706088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.706204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.709443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618897bd-3254-4fa6-afa3-fdccf815d6b2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.717224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.718834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:56.736288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:56.737424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:56.738390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:56.738390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:56.738390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:56.738390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:56.738390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 396/796] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:56.738390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.751052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.752244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.754286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.754585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.755242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.757063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.760740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:56.769194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:56.770120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 18:24:56.770935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:56.775896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 401/797] 10.222.0.49 () {56 vars in 1129 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:56.776808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.834467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:56.872997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5850d-5abc-49ba-b918-afdc633e5f14 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.883214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586bfe4-8393-4230-867b-ca0cef71d705 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:56.908458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.909341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.909625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.909957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.910723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:56.911023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.912387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.924256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:56.924892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.929712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.929811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 397/798] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:56.929911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.929911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:56.930040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:56.930108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:56.930108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b6590-cd48-4aec-9ddb-df302f456b4e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:56.945261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 402/799] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:56.945261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742034d-4406-4388-8242-01b6b2790dad None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:56.945261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742034d-4406-4388-8242-01b6b2790dad None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:56.945261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742034d-4406-4388-8242-01b6b2790dad None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:56.945261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 403/800] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:56.945261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742034d-4406-4388-8242-01b6b2790dad None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.018120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.043084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c860-112c-4fe9-b91d-7cf958de9800 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.053682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742034d-4406-4388-8242-01b6b2790dad None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.068484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.079056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.079667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.079938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.080202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.080899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:57.081195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.082556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.092174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 398/801] 10.222.0.49 () {56 vars in 1124 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:57.092174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.095921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.096579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.096888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.097159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.104679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.106174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fc8a9248-5814-4953-a5a6-7a06066e1097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:57.117348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:57.118259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:57.119177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 404/802] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:57.119674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.119879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.120151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.137951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:57.139067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:57.190547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6575483-1b92-4888-9567-5f542a35cb3a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.205628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 405/803] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:57.206442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.234081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.234729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.235232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.235232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.243179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.244603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.247345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c699f55f-4e6c-419a-a84d-4d563abe22de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:57.257575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:57.258665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:57.259620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 399/804] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:57.260186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e82581-54b0-4806-b309-b2a1142bf242 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.260186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e82581-54b0-4806-b309-b2a1142bf242 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.260409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e82581-54b0-4806-b309-b2a1142bf242 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.261643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 400/805] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:57.262309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e82581-54b0-4806-b309-b2a1142bf242 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.310625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732db1ba-b06a-4471-a1ce-aa0cad8ffba4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.312962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.344259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.344924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.345202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.345468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.346165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:57.346462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.347831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.352493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 406/806] 10.222.0.49 () {56 vars in 1124 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:57.353332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.362986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e82581-54b0-4806-b309-b2a1142bf242 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.398001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.398615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.398897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.399166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.399895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:57.400196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.401656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.413136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 401/807] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:57.413912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.456031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f23b6-d9da-4971-b54a-a8d733c28fda None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.471919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.491825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.492959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.493435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.493858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] PATH_INFO: `/v3/services/network` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.494676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.496044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-470d624e-a302-438b-87f1-8adb27936cba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:57.507854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:57.508933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:57.509762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 407/808] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:57.510963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.511265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.511427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.522480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a12191-05a9-45ca-aa57-8016f199dfd2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.528932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:57.530103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:57.554698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.556988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.557608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.557869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.558141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.558966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:57.559244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.560706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.578320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 402/809] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:57.579106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.598422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 408/810] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:57.599440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.688836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23015c69-1304-47c3-9e6c-66e538ae99bc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.695686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb34ce-358d-4a1b-8d77-b92c3171b881 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.706064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.723256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.723874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.724154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.724425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.725165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.726522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.728559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.729261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.729530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.729794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/e64871a43e564c9bbf7ebfdbd512a1f3/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.731676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=e64871a43e564c9bbf7ebfdbd512a1f3, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.733128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.746716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 409/811] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:24:57 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/e64871a43e564c9bbf7ebfdbd512a1f3/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:24:57.747756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.773714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 403/812] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:24:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:57.773714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:57.788635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:57.913301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90119-ce86-4dba-90c6-4a6b060005a1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.932963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dc510-9ffe-48a2-95ad-593d3ad6d9a4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:57.961761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.962313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.962583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.962849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.976629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:57.977578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:57.977852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:57.978122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:57.978800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:57.979094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:57.980463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:57.986901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 410/813] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:57.987532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.012951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.054501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.056106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.082517 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.098449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a335134-c2b5-421f-9fa6-47c1d4e52e1d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.134246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.135228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.135689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.136127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.146837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:24:58.151993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:58.153126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:24:58.154146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:58.156479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 411/814] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:58.157428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.164261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 404/815] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:24:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:24:58.165271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.239873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.273846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6a648-a830-4be2-8c98-39b22ae94f97 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.289566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdafc8b-e3e6-4297-a8e6-8e631eed1fe0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.309289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.309916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.310188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.310459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.311324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:58.311627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.313352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.325845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 405/816] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:58.326757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.334972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.336183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.336604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.337044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.338224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] RBAC: Adding query filter params (type=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:58.338668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.340782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.352500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 412/817] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:58.353871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.369262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.432081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36873465-9052-4f84-8302-7a89abf874a5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.467269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.467985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.468314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.468579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.478269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942a74f-2131-4611-b5a1-df91cf121d13 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.482825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.484638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:58.503575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:58.504513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:58.505420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:58.505420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:58.505420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:58.505420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:58.505420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.505420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 406/818] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:58.509812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.517893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.518672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.519043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.519419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.530173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.531816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:58.550553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:58.552056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:58.553978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:58.553978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:58.553978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:58.553978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:58.553978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 413/819] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:58.553978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.634291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.642408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9a7f7-4ebc-45f2-803c-d3a3d2d4f86e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.681107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cd127-85b7-4172-a11c-0b356118bad4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.685089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.685716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.685989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.686258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.687015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.688374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.716276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 407/820] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:24:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:24:58.717203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.743903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.744636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.745034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.745398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.746239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:58.746656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.748171 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.768302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:58.774849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 414/821] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:58.782178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.849369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038f19c-f71c-4463-9b0f-8bce6c46a3b2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:58.905380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:58.906576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:58.907118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:58.907598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:58.908513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:58.909095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:58.910776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:58.936194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 408/822] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:58.945893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:58.967317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af53c2-ac16-4fad-97e3-99d4bb374e4b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.019089 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.020484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.020953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.021845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.029781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:59.032730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.034281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:59.049992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:59.051105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:24:59.052166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 415/823] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:59.056901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.098300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:59.102552 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be707bf0-ce2c-49e9-9f5c-e40c3fff4b10 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.152334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.153621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.154046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.154724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.163794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.166338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.174388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63a1c2-dc94-41f3-a7e7-3b080a0d8fc4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:24:59.179727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:59.180641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:59.181707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 409/824] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:24:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:24:59.186883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.214017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.215173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.215610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.216015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.217491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:59.218037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.219625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.236421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 416/825] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:59.243894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.305209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:59.310027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6beea1-1926-4dfc-99b3-b854bb63ee0f None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.345393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.346070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.346354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.346619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.347411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:59.347710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.349168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.358586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed53cb-d06c-48d6-9cda-143c7d874154 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.367626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 410/826] 10.222.0.49 () {56 vars in 1149 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:24:59.372746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.397485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:59.409814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.411205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.411931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.412408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.413525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=9958737194be4488a1e09e6e48557d75) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:59.414245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.416958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.433046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 417/827] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/role_assignments?user.id=9958737194be4488a1e09e6e48557d75&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:59.494738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c41f26-1b16-4334-9397-5cb96f9ffc79 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.534338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.535003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.535278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.535549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.547672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.549270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:24:59.565906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:59.566833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:59.567717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:24:59.567717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:24:59.567717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:24:59.567717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:59.567717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 411/828] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:59.573105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.680124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff7875-a343-49d7-bafe-8e267bde0ff0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.681663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:59.716311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.716969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.717296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.717565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.718339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:24:59.718640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.720010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125d306-26cf-494c-bf7e-082bed8a1bd5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.732656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 418/829] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:24:59.738365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.810086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:24:59.843317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b65d90-ff22-45df-9ccf-e08fa29efeb4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:24:59.877777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:24:59.878479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.878763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.879036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.882996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e739a-80f0-43e3-83b9-d052001269cf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.883273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e739a-80f0-43e3-83b9-d052001269cf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.883544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e739a-80f0-43e3-83b9-d052001269cf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.886170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 419/830] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:24:59.886716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:24:59.888228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:24:59.890759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5888-846e-4f35-8f94-f30accfe4110 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.891400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5888-846e-4f35-8f94-f30accfe4110 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.891674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5888-846e-4f35-8f94-f30accfe4110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:24:59.900905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:24:59.901850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:24:59.902738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 412/831] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:24:59.908092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:24:59.911601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a82c5888-846e-4f35-8f94-f30accfe4110 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:24:59.912787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a82c5888-846e-4f35-8f94-f30accfe4110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:24:59.991238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 420/832] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:24:59.992581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f86a7-85fe-4d14-8193-42730d8357b0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:24:59.992887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f86a7-85fe-4d14-8193-42730d8357b0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:24:59.993152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f86a7-85fe-4d14-8193-42730d8357b0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:24:59.996793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 421/833] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:24:59.999719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.000002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.000277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.019407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edc4d-aa8a-415d-9674-1adbd7902b3b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.023456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:00.024650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:00.049078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:00.056663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.057324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.057601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.057873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.058631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:00.058936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.060378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.072748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 413/834] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:00.077038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.107423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 422/835] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:24:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:00.108299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.133380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:00.193460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1666-50cb-44cc-aab5-23e22be1f989 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.217212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8032af-8bde-4453-9560-5917cc2782bd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.236794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.237459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.237761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.238018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.249603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:00.254525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:00.255469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:00.256678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 414/836] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:00.258049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.263345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.264030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.264309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.264580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/4242cff2780f478099110d0586ad67b3/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.280530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=4242cff2780f478099110d0586ad67b3, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.282102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.298143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 423/837] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:25:00 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/4242cff2780f478099110d0586ad67b3/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:25:00.299270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.326234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:00.380021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:00.401144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31503ae7-2ea9-47f2-b245-7295f1ea8794 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.420916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7070f75-ea89-441a-8636-b97e8edb7c2e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.439564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.440497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.441025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.441491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.442372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.443982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:00.454393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:00.455474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:00.456298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:00.463790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.464378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 415/838] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:00.464486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.464781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.465060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.465791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:00.466090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.466437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.467611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.478909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 424/839] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:00.486894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.594775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1c545-8e63-449f-9900-c11edab17322 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.613403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d41c1c-49ff-4f3c-a90f-c13829c58cf7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.631372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.632002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.632277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.632544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.632883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:00.635063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:00.646872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.649094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.657831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.658231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.658614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.658963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.659760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:00.660178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.661698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.671398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 416/840] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:00.671554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:00.672461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:00.673365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:00.673365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:00.673365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:00.673365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:00.673593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.677056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 425/841] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:00.680597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.786497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6121e-2e96-40e9-9ca0-25c713d4fd24 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.817131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66c76c-92e2-4661-a420-eaf02ef10e9a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:00.822161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.822798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.823075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.823345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.824030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:00.824329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.825735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.836858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 417/842] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:00.863136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:00.863779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:00.864054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:00.864325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:00.865072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:00.865373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:00.866776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:00.881669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 426/843] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:00.887284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:00.935779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:01.003974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40551048-acd7-4ad0-80d7-cb1310c166f9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:01.042433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:01.043125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.043405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.043743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.052024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:01.053630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:01.066133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:01.067110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:01.068030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 418/844] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:01.075479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.123770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:01.179217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b6ce6-828a-441d-bdf5-367643d0d7bf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:01.213661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:01.214292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.214627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.214899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.224126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:01.224690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:01.225978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:01.244747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 427/845] 10.222.0.49 () {56 vars in 1135 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:01.253865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.359692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a4020-0267-4aab-8d0f-4ba75d7b761a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:01.361626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:01.396362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce775dd-3d83-471d-bf4f-965a8fa18fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:01.397022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce775dd-3d83-471d-bf4f-965a8fa18fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.397300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce775dd-3d83-471d-bf4f-965a8fa18fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.397567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce775dd-3d83-471d-bf4f-965a8fa18fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.466833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce775dd-3d83-471d-bf4f-965a8fa18fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:01.468273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce775dd-3d83-471d-bf4f-965a8fa18fb0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:01.571588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 419/846] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:25:01.577343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.590955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bc36c-6d8b-482a-ac53-0de4f0254ee6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.591230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bc36c-6d8b-482a-ac53-0de4f0254ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.591499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bc36c-6d8b-482a-ac53-0de4f0254ee6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.593167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 420/847] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:01.598022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.598295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.598561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.605753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:01.622890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:01.624223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:01.688564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab79227-5a76-4f78-863f-0a97ec678d10 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:01.692015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:01.712608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 421/848] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:01.734137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:01.734347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.735164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.735635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.736122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.746978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:01.748636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:01.764317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:01.765293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:01.766191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:01.766191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:01.766191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:01.766191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:01.768351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 428/849] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:01.773958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.836969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a3b7f-8ba9-4566-a240-b9b97bcba2f9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:01.869160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:01.873754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:01.874438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.874725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.875426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.876068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:01.877986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:01.882876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3941-e8e0-45a3-bb28-e0bda82163ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:01.889786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:01.890761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:01.891584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 422/850] 10.222.0.49 () {56 vars in 1132 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:01.897584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.928889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:01.930038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:01.930867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:01.931282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:01.932387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:01.932848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:01.935254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:01.955315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 429/851] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:01.957229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:01.980548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:02.034147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d134b8-700f-4987-bfd3-97d6b4ccb59c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.072065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.072510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.072805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.073265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.074062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:02.074462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:02.076034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:02.079355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd653c8-3de6-4fc9-9646-25dae3d2d810 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.086320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 423/852] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:02.087659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:02.115007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.115921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.116344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.116746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.123818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:02.125523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:02.137738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:02.138642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:02.139549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:02.146499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 430/853] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:02.147594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:02.161133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:02.191805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebe58e-2b86-41d0-b8bf-3515d9d89afb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.227147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.227848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.228192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.228815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.229492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:02.229713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:02.231157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:02.244812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:02.245794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 424/854] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:02.262601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704970f-afeb-4c2c-a9a2-8150e0836785 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.300859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.301425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.301701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.301969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.302734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:02.303036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:02.304501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287d21-2d5c-4566-a8b4-d6d7c3caa07e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:02.318177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 431/855] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:02.324262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:02.404790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b899b-f432-4ec0-8fb3-1edb33f49997 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.404790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b899b-f432-4ec0-8fb3-1edb33f49997 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.404790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b899b-f432-4ec0-8fb3-1edb33f49997 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.404790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 432/856] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:02.406289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.406563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.406830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.427581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:02.428771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:02.441062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373490d-7be2-4b48-8926-872a99bf5302 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.468950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:02.481622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.482429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.482795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.483195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.484139 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=e64871a43e564c9bbf7ebfdbd512a1f3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:02.484532 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:02.486095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:02.501100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 425/857] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:02 2022] GET /identity/v3/role_assignments?user.id=e64871a43e564c9bbf7ebfdbd512a1f3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:02.512442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 433/858] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:02.535756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:02.672734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed3b8-c94a-46c2-9449-795d44ba9be7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.707841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.708485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.708774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.709051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.719734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:02.725270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:02.726184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:02.727004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 426/859] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:25:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:02.732368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:02.799042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:02.835054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2e80c-a264-492c-8a85-802aed4451a2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:02.869969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:02.870597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.870886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.871147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.871862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:02.872163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:02.873569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff26a-1566-4e65-a695-6bfc9d37de3b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:02.882702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 434/860] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:25:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:02.889980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:02.941143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:02.990163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49221bb2-fad2-450d-a4a5-3e5f91d59e19 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.990500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49221bb2-fad2-450d-a4a5-3e5f91d59e19 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.990711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49221bb2-fad2-450d-a4a5-3e5f91d59e19 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.992318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 435/861] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:02.997081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:02.997365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:02.997640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:02.999189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a440ca-891b-4220-804d-b1de1d1d3886 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:03.016006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:03.017292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:03.037138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:03.037877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:03.038152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:03.038515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:03.049824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:03.051276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:03.067832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:03.068876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:03.069864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:03.069864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:03.069864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:03.069864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:03.071303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 427/862] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:03.075044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:03.092743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 436/863] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:03.114841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:03.175945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:03.198252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:03.201463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa775e-9751-4be5-b00b-b0c81b29b34c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:03.238438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db8367-05d1-46fa-ace8-28a23652426e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:03.244916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:03.245933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:03.246303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:03.246669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:03.247924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:03.248326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:03.250337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:03.268850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 428/864] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:03.271326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:03.282870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:03.284062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:03.284436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:03.284827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:03.286005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:03.287728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:03.316416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 437/865] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:03 2022] POST /identity/v3/services => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:03.388406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c578811-6046-45e3-8c6e-9ca10d8ea290 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:03.433789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:03.434455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:03.434720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:03.434985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:03.443635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:03.445222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:03.457863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:03.458820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:03.459788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 429/866] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:25:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:03.465038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:03.576102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d72b0d-e0a9-4ce3-a3b2-a3b61873cf9b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:03.593864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:03.611964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:03.612591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:03.612892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:03.613165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:03.613955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:03.614255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:03.615678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:03.634200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 438/867] 10.222.0.49 () {56 vars in 1149 bytes} [Sun Jul 31 18:25:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:03.639772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:03.659096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:03.744580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeaf22-6c34-417c-a466-4d4f411007b7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:03.779200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:03.779825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:03.780100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:03.780427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:03.849792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:03.851253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:03.869084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:03.952388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 430/868] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:03 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:25:03.958084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.061028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb347b-de76-4c9a-af17-4742795af3b7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.094884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.095529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.095802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.096064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.109157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:04.111380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:04.130435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:04.131734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:04.133237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:04.133237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:04.133237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:04.133237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:04.133237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 439/869] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:04.138166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.240790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd65d0-189a-410c-94a4-6307056ee27d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.258554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:04.276918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.277542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.277875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.278149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.278850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:04.279138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:04.280539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4924b-f15b-430d-84d6-f181085e512e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:04.292737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 431/870] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:04.298400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.330661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:04.377456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8b43-3dd7-4ca2-b113-5941300c2a30 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.377890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8b43-3dd7-4ca2-b113-5941300c2a30 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.378345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8b43-3dd7-4ca2-b113-5941300c2a30 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.381377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 432/871] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:04.385469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.385855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.386231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.407897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:04.409325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:04.413161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a662e9-1f31-40e3-815d-6301aaf07f14 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.459052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.460574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.461096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.461980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.471692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:04.473975 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:04.489649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:04.490587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:04.491494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:04.493941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 440/873] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:04.498716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 433/873] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:04.500007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.516451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.610508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08614-99f0-4f2b-b0c8-1e37361e49e6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.610896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:04.620717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:04.628035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16f09d-7501-4dae-8875-64ea014fb529 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.654954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.655777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.656062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.656350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.657115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:04.657418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:04.658840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da644c-19d8-4f2b-99d0-645c369b339e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:04.664633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28fa6-ed73-4d7e-84e4-f0f0986c4077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.665288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28fa6-ed73-4d7e-84e4-f0f0986c4077 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.665944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28fa6-ed73-4d7e-84e4-f0f0986c4077 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.665944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28fa6-ed73-4d7e-84e4-f0f0986c4077 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.666520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec28fa6-ed73-4d7e-84e4-f0f0986c4077 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:04.668067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec28fa6-ed73-4d7e-84e4-f0f0986c4077 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:04.676329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 434/874] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:04.677515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d972-e02d-4307-94f6-7a5eb625cc42 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.677873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d972-e02d-4307-94f6-7a5eb625cc42 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.678062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d972-e02d-4307-94f6-7a5eb625cc42 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.679589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 435/875] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:04.680276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d972-e02d-4307-94f6-7a5eb625cc42 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.694351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 441/876] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:04 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:04.694351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.694351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.694351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.718710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:04.720043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:04.808353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 442/877] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:04.820277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d972-e02d-4307-94f6-7a5eb625cc42 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.827218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:04.862420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.863336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.863765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.864175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.865276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=4242cff2780f478099110d0586ad67b3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:04.865837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:04.867473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:04.884845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 436/878] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:04 2022] GET /identity/v3/role_assignments?user.id=4242cff2780f478099110d0586ad67b3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:04.919605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:04.952289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e1d64-5a2d-42d6-912c-c2f87946a172 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:04.995849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:04.996697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:04.997008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:04.997316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:04.998473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:05.000215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:05.011028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:05.012063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:05.012917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 443/879] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:05.019341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:05.065128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:05.142674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b42262-b5eb-411c-aebb-397c7b4c019e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:05.181257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:05.182244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:05.182718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:05.183204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:05.184087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:05.184577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:05.186206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fbe1e-1a2f-438c-bc60-1d9714a6aefe None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:05.191708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 437/880] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:05.197157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:05.303928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5992056-285a-477a-9f88-82063b1d8ac5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:05.339252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:05.339935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:05.340210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:05.340480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:05.341292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:05.341598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:05.343135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787194b-a633-4b65-b569-26418acd6f14 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:05.348252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 444/881] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:05.541810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c197df5-3c5f-4410-8c25-1bc10c4becb5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:05.542309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c197df5-3c5f-4410-8c25-1bc10c4becb5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:05.542309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c197df5-3c5f-4410-8c25-1bc10c4becb5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:05.543920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 438/882] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:05.549581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:05.550052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:05.550534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:05.559243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:05.575018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:05.576208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:05.645031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 445/883] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:05.667076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c197df5-3c5f-4410-8c25-1bc10c4becb5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:05.764717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:05.773233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c197df5-3c5f-4410-8c25-1bc10c4becb5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:05.807554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f4171-704f-4363-8702-985a33768499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:05.808109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4171-704f-4363-8702-985a33768499 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:05.808380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4171-704f-4363-8702-985a33768499 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:05.808645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4171-704f-4363-8702-985a33768499 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:05.809414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f4171-704f-4363-8702-985a33768499 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:05.810771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f4171-704f-4363-8702-985a33768499 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-281f4171-704f-4363-8702-985a33768499 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:05.823921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:05.824923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:05.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 439/884] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:05.831118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:05.938535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015d44f-b578-48b8-a030-d3342dbc83f2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:05.969550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:05.975780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:05.976881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:05.977357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:05.977840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:05.978738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:05.979247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:05.980885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:05.987095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 446/885] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:05.994142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f4171-704f-4363-8702-985a33768499 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:06.101025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f4171-704f-4363-8702-985a33768499 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:06.135587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:06.136201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:06.136471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:06.136738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:06.137439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:06.137732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:06.139087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:06.144387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 440/886] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:06.150563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:06.267398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19c709-efed-456b-bdb8-9c9268ec5a6c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:06.302678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:06.303304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:06.303574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:06.303837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:06.304905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] RBAC: Adding query filter params (interface=public, service_id=101c36b47dd64b20a6d5c66e304b753d, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:06.305441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:06.307440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e6d9-139f-4e09-a108-800dcff70135 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:06.313180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 447/887] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:25:06 2022] GET /identity/v3/endpoints?service_id=101c36b47dd64b20a6d5c66e304b753d&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:06.318757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:06.345853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:06.422603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cf220-3e90-4028-afac-b75d47043855 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:06.460731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a5f61-e577-414d-bc46-f26cb852e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:06.461452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a5f61-e577-414d-bc46-f26cb852e25a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:06.461726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a5f61-e577-414d-bc46-f26cb852e25a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:06.462044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a5f61-e577-414d-bc46-f26cb852e25a None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:06.462698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a5f61-e577-414d-bc46-f26cb852e25a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:06.464055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a5f61-e577-414d-bc46-f26cb852e25a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:06.471322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 441/888] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:25:06 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:25:06.903120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53efd3-d5f5-4b73-889d-8f1cdf5e9637 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:06.903527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53efd3-d5f5-4b73-889d-8f1cdf5e9637 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:06.904119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53efd3-d5f5-4b73-889d-8f1cdf5e9637 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:06.906185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 448/889] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:06.912100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537f942-7bf4-40e5-910b-948c68afc71f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:06.913164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537f942-7bf4-40e5-910b-948c68afc71f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:06.913638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537f942-7bf4-40e5-910b-948c68afc71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:06.932511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3537f942-7bf4-40e5-910b-948c68afc71f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:06.937115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3537f942-7bf4-40e5-910b-948c68afc71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:07.009488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 442/890] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:07.030984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53efd3-d5f5-4b73-889d-8f1cdf5e9637 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.095481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.128238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aca5be-73b1-43f6-adba-a78eed51dc3c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.128926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aca5be-73b1-43f6-adba-a78eed51dc3c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.129403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aca5be-73b1-43f6-adba-a78eed51dc3c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.131597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 443/891] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:07.136624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.137158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.137630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.140398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53efd3-d5f5-4b73-889d-8f1cdf5e9637 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.157398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:07.159382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:07.178545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.179498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.179897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.180302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.181119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.182625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.189146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:07.192000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:07.192953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:07.193786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:07.196613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 449/892] 10.222.0.49 () {56 vars in 1132 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:07.202888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.245204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 444/893] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:07.269643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.341495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201db043-5b75-4eee-a068-4c1e26eead6e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.375228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.379671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b1dbe-1709-4d89-936f-96d8bdd761c0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.389901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.390792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.391671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.392114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.393346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:07.393658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.395048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.400251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 450/894] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:07.411616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.431835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2ae4b-d288-40a6-95af-86d72c3ea032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.432481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2ae4b-d288-40a6-95af-86d72c3ea032 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.432754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2ae4b-d288-40a6-95af-86d72c3ea032 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.433054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2ae4b-d288-40a6-95af-86d72c3ea032 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.443659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2ae4b-d288-40a6-95af-86d72c3ea032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e0a2ae4b-d288-40a6-95af-86d72c3ea032 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:07.448570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:07.449516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:07.450345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 445/895] 10.222.0.49 () {56 vars in 1120 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:07.451031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b44912-f413-4258-b83f-7247e9a15042 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.451431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b44912-f413-4258-b83f-7247e9a15042 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.451521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b44912-f413-4258-b83f-7247e9a15042 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.456599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 446/896] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:07.457379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b44912-f413-4258-b83f-7247e9a15042 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.466495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.535526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05c98b-a9d5-40d3-8672-bf42ec6308a2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.564184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b44912-f413-4258-b83f-7247e9a15042 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.571598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c157be-b18a-446d-ba61-95331478877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.572626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c157be-b18a-446d-ba61-95331478877d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.572951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c157be-b18a-446d-ba61-95331478877d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.573225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c157be-b18a-446d-ba61-95331478877d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.573902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c157be-b18a-446d-ba61-95331478877d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:07.574555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c157be-b18a-446d-ba61-95331478877d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.576255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c157be-b18a-446d-ba61-95331478877d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.584934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 451/897] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:07.586373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.586656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.586922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.607763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:07.608001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.608555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.608875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.609044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:07.609357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.610086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:07.610378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.611733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.620262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 447/898] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:07.620878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.661232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.682166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 452/899] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:07.689879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.693417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.730014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6127960-7102-4aca-9369-ed43be0eaa3a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.768037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.768659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.768960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.769232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.770365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] RBAC: Adding query filter params (interface=public, service_id=99111cff49d9469bb92e906b823d5546, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:07.770678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.772381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.784902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 448/900] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/endpoints?service_id=99111cff49d9469bb92e906b823d5546&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:07.785810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.798321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115694b-c625-4ffd-9134-d5a9a2aa9987 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.834504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.835172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.835445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.835714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.845767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.847211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:07.862681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:07.863574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:07.864502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:07.864502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:07.864502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:07.864502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:07.864502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 453/901] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:07.864800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.894728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e78e8-0437-4402-9bb0-488dd9e4bba0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:07.900216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.927089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:07.934456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:07.935050 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:07.935397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:07.935664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:07.936480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:07.937910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:07.961056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 449/902] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:07 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:07.961909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:07.973934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48646ab3-ecbe-4d28-9b17-dc917c02b3da None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.037301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.038582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.039085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.039587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.041072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.044183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.053781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 454/903] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:25:08.090599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2428a-9fae-4ed5-8d62-98d12ba6bc12 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.132184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.132878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.133160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.133432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.134159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:08.134469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.135887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.150307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 450/904] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:08.155847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.262387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:08.270557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c237d0-e61c-4650-b26e-755fb484d68f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.306023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.306678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.306955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.307217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.315253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.316701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-664988b2-f704-4291-8955-5786d60e40dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:08.328185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:08.329202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:08.330161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 455/905] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:08.335430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.396280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:08.451581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad57991-7991-43f4-83b4-9e4c3c5ccf07 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.489051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2180-d395-4cd8-b79c-17f0f3994342 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.489563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.490112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.490325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2180-d395-4cd8-b79c-17f0f3994342 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.490425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.490640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.491063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2180-d395-4cd8-b79c-17f0f3994342 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.491439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:08.491734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.493290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.493513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 456/906] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:08.498891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.499262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.499524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.516637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 451/907] 10.222.0.49 () {56 vars in 1149 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:08.522441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.523035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:08.524228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:08.594028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 457/908] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:08.615758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.642721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e44c5-73ee-4d0c-889c-1dec5525b760 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.655958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:08.687095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.688005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.688282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.688552 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.715373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:08.737845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272325a-afea-4336-bc3a-7a9b8402e02a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.775554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.776564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.777064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.777571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.778529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.780184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.780660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.782555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:08.790285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:08.791268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 18:25:08.792078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:08.800324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 458/909] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:08.801669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.905041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 452/910] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:25:08.910101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.914721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4bfb-a21d-4a9a-9580-1bd24f31794f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:08.922427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:08.953670 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:08.954652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:08.955163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:08.955627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:08.956570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:08.957168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:08.958779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:08.964438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 459/911] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:08.969723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:08.986490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:09.032313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f45030-0ba6-4a07-a72b-c80bed6fbfc2 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.071545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.072190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.072520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.072725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.085856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.087318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.098486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b6d10-4549-4871-9fdd-c141dcd87b06 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:09.111230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:09.112146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:09.113094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:09.113094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:09.113094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:09.113094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:09.118346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 453/912] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:09.119224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:09.142051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.143562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.144176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.144723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.145869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:09.146347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.148566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.156577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 460/913] 10.222.0.49 () {56 vars in 1114 bytes} [Sun Jul 31 18:25:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:09.162709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:09.195391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:09.243727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dcbc1-0d13-4725-94dd-198337970e42 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.258997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:09.283880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2e009-1e3d-447c-b0ae-3edf74524c13 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.284493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.285527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.285981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.287217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.288097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:09.288577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.290224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.305594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 454/914] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:09.315699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:09.333072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.334014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.334468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.334913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.335828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.337448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.369365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 461/915] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:25:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:09.437726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f868dd-1603-4a9c-8553-8b4a3e7c9658 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.465937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:09.485917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.486628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.486907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.487173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.495380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.496914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:09.511361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:09.512327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:09.513268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 455/916] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:09.520316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:09.628327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b1e98-68a6-4e89-b901-5e1d63457e65 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.659214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:09.665464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.666002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.666290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.666919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.667851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:09.668163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.669640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e5b1c-958b-4c79-8503-f873813cf937 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.683086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 462/917] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:09.689134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:09.792580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e9fde-3c53-49da-b887-55670f0e23ec None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:09.827334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:09.827976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:09.828253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:09.828526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:09.829354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] RBAC: Adding query filter params (role.id=f25d6c9b45554746b1730a94e6a72f2e, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=4242cff2780f478099110d0586ad67b3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:09.829661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:09.831210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edf9b9-d180-46d5-901b-b3675a7321e5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:09.837511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 456/918] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:09 2022] GET /identity/v3/role_assignments?user.id=4242cff2780f478099110d0586ad67b3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=f25d6c9b45554746b1730a94e6a72f2e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:10.065837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee4737-c075-46ed-b0af-d1f78fc50b8d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.066365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee4737-c075-46ed-b0af-d1f78fc50b8d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.066780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee4737-c075-46ed-b0af-d1f78fc50b8d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.068784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 463/919] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:10.074244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4f0a2-836d-4165-af39-f24b60b84e44 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.075085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4f0a2-836d-4165-af39-f24b60b84e44 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.075556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4f0a2-836d-4165-af39-f24b60b84e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.094518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-edc4f0a2-836d-4165-af39-f24b60b84e44 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:10.096078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-edc4f0a2-836d-4165-af39-f24b60b84e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:10.133609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:10.169758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 457/920] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:10.192109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee4737-c075-46ed-b0af-d1f78fc50b8d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.298634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75d30b-190f-4da1-9f71-369c91396214 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.298999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75d30b-190f-4da1-9f71-369c91396214 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.299187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75d30b-190f-4da1-9f71-369c91396214 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.300727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 458/921] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:10.303166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee4737-c075-46ed-b0af-d1f78fc50b8d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.305948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.306459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.306924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.325520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:10.327136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:10.339222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.339867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.340161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.340417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.341082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.342551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:10.352184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:10.353383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:10.354250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 464/922] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:10.360875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.404689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 459/923] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:10.427229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:10.427766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.492078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344472e8-c31a-4b70-bb08-6286930c1112 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.540870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.542326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.542810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.543715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.544883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:10.545385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.547810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.554592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 465/924] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:10.559014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc627-4f78-4fd9-8837-dc004539bc9f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.560083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.562304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:10.598603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.599150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.599421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.599689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.600314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.601704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:10.610980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:10.611945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 18:25:10.612852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:10.616910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 460/925] 10.222.0.49 () {56 vars in 1132 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:10.617886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.680608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce24db6-2f43-48b6-aa20-727e2064f8e5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.702429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:10.718133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b5c43-73b6-4dbc-929d-e4a4a8a773a3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.719327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.720322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.720819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.721318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.722183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:10.722678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.724504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.735071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 466/926] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:10.736887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.752282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.752950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.753286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.753556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.754225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:10.754522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.755877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.766038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 461/927] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:10.766854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.796947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:10.850025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953b9a3-8cfd-41e5-9968-804b62c6f5d2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.867205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972bea-27f4-455d-93ab-1cfd0cc99e36 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:10.888291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.889289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.889858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.890512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.891590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] RBAC: Adding query filter params (interface=public, service_id=92d3d71cd8e24bc6a3c519b8647cf8dc, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:10.892070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.893970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.900590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 467/928] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/endpoints?service_id=92d3d71cd8e24bc6a3c519b8647cf8dc&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:10.902373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:10.902998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:10.903269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:10.903536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:10.904214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:10.904518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:10.906010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:10.906917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.915999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 462/929] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:10.917018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:10.963089 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:11.027284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c1bef-1fde-436b-8d5c-5614787c9d21 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:11.042077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:11.050889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225ee8f-64da-4b20-abb4-1dd91858d92c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:11.070097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbf31a-4ea7-48b0-a676-f5a66228da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:11.070876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf31a-4ea7-48b0-a676-f5a66228da87 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.071162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf31a-4ea7-48b0-a676-f5a66228da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.071433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbf31a-4ea7-48b0-a676-f5a66228da87 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.072084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbf31a-4ea7-48b0-a676-f5a66228da87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:11.073670 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbf31a-4ea7-48b0-a676-f5a66228da87 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:11.083064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 468/930] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:25:10 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:25:11.091642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cff60-5e87-48cc-ac5e-70370accad31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:11.093138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cff60-5e87-48cc-ac5e-70370accad31 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.093619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cff60-5e87-48cc-ac5e-70370accad31 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.094087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cff60-5e87-48cc-ac5e-70370accad31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.095127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cff60-5e87-48cc-ac5e-70370accad31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:11.096840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cff60-5e87-48cc-ac5e-70370accad31 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:11.123661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 463/931] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:25:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:11.413808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9727e1-eb83-4c0a-8f6c-c1f4802c9496 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.414183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9727e1-eb83-4c0a-8f6c-c1f4802c9496 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.414365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9727e1-eb83-4c0a-8f6c-c1f4802c9496 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.415932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 469/932] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:11.421508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.421788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.422058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.440308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:11.441517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:11.518341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 464/933] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:11.541106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9727e1-eb83-4c0a-8f6c-c1f4802c9496 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:11.647481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9727e1-eb83-4c0a-8f6c-c1f4802c9496 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:11.661036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:11.682603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:11.683148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.683422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.683690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.684382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:11.685860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:11.688534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING keystone.server.flask.common [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 31 18:25:11.708184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 18:25:11.708184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 31 18:25:11.708184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [parameters: {'id': '0bb55fe0ddd04d02ad12771348852e1e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 31 18:25:11.708184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bb55fe0ddd04d02ad12771348852e1e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bb55fe0ddd04d02ad12771348852e1e. Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 18:25:11.715683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 31 18:25:11.716471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Jul 31 18:25:11.717650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 18:25:11.719156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 18:25:11.720793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 18:25:11.721685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': '0bb55fe0ddd04d02ad12771348852e1e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:11.722562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:11.723332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bb55fe0ddd04d02ad12771348852e1e. Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:11.724242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 470/934] 10.222.0.49 () {60 vars in 1140 bytes} [Sun Jul 31 18:25:11 2022] POST /identity/v3/projects => generated 262 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:11.724815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:11.811992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:11.835615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2cf6-0f1e-4ae8-9b19-85d5abba7207 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:11.870687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:11.871379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.871734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.872002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.879018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:11.880464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6f40a286-7112-42fe-a463-d1919ef6d115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:11.892304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:11.894477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:11.896637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 465/935] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:11.900086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:11.931514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a896d17-2406-4557-878a-d222349bca58 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.931899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a896d17-2406-4557-878a-d222349bca58 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.932355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a896d17-2406-4557-878a-d222349bca58 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.935232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 466/936] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:11.939179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:11.939548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:11.939890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:11.958511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:11.959802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:12.008725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e7c02-aacb-478d-be8b-27f58e952743 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:12.040916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 467/937] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:12.050781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:12.051413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:12.051685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:12.051959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:12.052677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:12.053019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:12.054661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:12.064429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:12.076150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 471/938] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:12.139788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:12.185383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40236b79-e28c-446e-b6fb-43d77d013e91 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:12.226965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:12.227630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:12.227905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:12.228177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:12.240939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:12.245946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:12.246877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:12.247714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 468/939] 10.222.0.49 () {56 vars in 1120 bytes} [Sun Jul 31 18:25:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:12.253611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:12.301153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:12.375497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca797912-9dd1-455b-b20d-d1d57002506a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:12.411589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:12.412216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:12.412494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:12.412782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:12.413503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:12.413805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:12.415228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:12.427251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 472/940] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:25:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:12.429770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:12.534397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f3230-a021-4495-83ff-43d1461d8a56 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:12.560296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:12.572255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:12.573205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:12.573680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:12.574149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:12.585005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:12.586673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:12.601675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:12.602653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:12.603638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:12.603638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:12.603638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:12.603638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:12.605566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 469/941] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:12.611183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:12.706302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:12.715912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02d6ca-3781-45b7-91eb-d0ab40626dff None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:12.753336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:12.753964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:12.754235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:12.754551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:12.755230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:12.755524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:12.757271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:12.769881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 473/942] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:12.775425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:12.878726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c431604-df5a-41ad-b407-a245199d611c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:12.913701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:12.914340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:12.914620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:12.914884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:12.922703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:12.924141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:12.935903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:12.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:12.938142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 470/943] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:25:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:12.945903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.051173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b653c2b-276e-4f49-abd7-65544c4b58fb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.085434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.086062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.086332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.086595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.087446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:13.087741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.089260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045c402-fb2b-4370-97d3-31b30fc795ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.107453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 474/944] 10.222.0.49 () {56 vars in 1149 bytes} [Sun Jul 31 18:25:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:13.113838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.155361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.183494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5be09-6ca1-44a9-bcea-41eadf1d48d0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.183800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5be09-6ca1-44a9-bcea-41eadf1d48d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.184046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5be09-6ca1-44a9-bcea-41eadf1d48d0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.185631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 475/945] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:13.190444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883a14d-ea7d-4061-98f2-76124027a1c8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.190728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883a14d-ea7d-4061-98f2-76124027a1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.190996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883a14d-ea7d-4061-98f2-76124027a1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.209840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2883a14d-ea7d-4061-98f2-76124027a1c8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:13.211632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2883a14d-ea7d-4061-98f2-76124027a1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:13.217092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.227571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007c51-0dec-4fdb-b2ff-fd7e462f1298 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.267678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.268362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.268640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.268948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.279283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.280754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c43a530f-e237-4868-aed7-021c0d74ca39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:13.296182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:13.297092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:13.298702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:13.298702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:13.298702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:13.298702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:13.298702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 471/946] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:13.298993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcb7be-3fd1-447a-9aba-c34bab3a171a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.299223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcb7be-3fd1-447a-9aba-c34bab3a171a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.299536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcb7be-3fd1-447a-9aba-c34bab3a171a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.300954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 472/947] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:13.305504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcb7be-3fd1-447a-9aba-c34bab3a171a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.306701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 476/948] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:13.307886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.308186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.308438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.326335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:13.327617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:13.396562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 477/949] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:13.398301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.414932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.426496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcb7be-3fd1-447a-9aba-c34bab3a171a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.461144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.461760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.462029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.462291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.462964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:13.463256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.464604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.478448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.481037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 473/950] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:13.481999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.515072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91d6cc-dbff-4654-a12f-79a49741ecbe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.552889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.553536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.553813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.554081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.554702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.556089 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:13.565140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:13.566214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:13.567058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 478/951] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:13.567890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.606295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221af7c-2cf4-47d3-9b4d-3ca0a4b2bd90 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.641808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.642812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.643092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.643366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.643989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.645396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:13.655072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:13.656086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:13.656979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 474/952] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:13.657770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.675529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.679880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1780a80-d975-4623-b180-c3a97a30482d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.710021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.715501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.716512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.716809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.717083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.724040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.725535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:13.737091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:13.738165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 479/953] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:13.739124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.766272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80800847-e5d5-45c9-8ea0-7420d81db31b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.802654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.803298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.803577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.803853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.804534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:13.804864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.806249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.811178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 475/954] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:13.811987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.853113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0381b2-e947-48dd-b242-0fac5d17e404 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.888702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.889375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.889774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.890044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.890721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:13.891021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.892402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.895467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.897771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 480/955] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:13.898803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:13.922760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12e25a-0cab-4a1a-9de8-5092809da5b9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:13.955625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:13.958799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:13.959605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:13.959894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:13.960166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:13.960956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:13.961269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:13.962676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:13.974850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 476/956] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:13.975647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:14.005016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3ff1-c86b-4599-bee3-6d2d8bca30a3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.039987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.040583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.040898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.041176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.041835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:14.042120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.043488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:14.048444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 481/957] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:14.049213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:14.095433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f054cd7-a9c0-488d-812d-52ebcfa37d0b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.134220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.134775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.135045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.135313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.136093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:14.136412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.137892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:14.153002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 477/958] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:14.153002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:14.155986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:14.164541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab433f-dbeb-4b6a-8a2e-a29332169424 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.188870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:14.204959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0464f5d-bc97-4210-ba93-5fb925f9ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.205706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0464f5d-bc97-4210-ba93-5fb925f9ae15 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.205922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0464f5d-bc97-4210-ba93-5fb925f9ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.206190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0464f5d-bc97-4210-ba93-5fb925f9ae15 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/4242cff2780f478099110d0586ad67b3/roles/f25d6c9b45554746b1730a94e6a72f2e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.208481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0464f5d-bc97-4210-ba93-5fb925f9ae15 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=4242cff2780f478099110d0586ad67b3, role_id=f25d6c9b45554746b1730a94e6a72f2e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.210013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0464f5d-bc97-4210-ba93-5fb925f9ae15 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:14.238002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 482/959] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:25:14 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/4242cff2780f478099110d0586ad67b3/roles/f25d6c9b45554746b1730a94e6a72f2e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:25:14.299753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32689b83-08e1-4164-8e5c-a2b7935c5c94 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.314361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac06fa-7d7d-47e6-8c05-4ff24772ff36 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.314702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac06fa-7d7d-47e6-8c05-4ff24772ff36 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.314917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac06fa-7d7d-47e6-8c05-4ff24772ff36 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.320656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 483/960] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:14.322509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.322789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.323065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.341363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.341987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.342255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.342559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.343314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.343834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:14.344684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:14.345114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:14.375925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 478/961] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:25:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:14.441408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 484/962] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:14.465207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:14.531191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:14.575757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ffe60-b546-4a53-8e7e-f3e181e06c07 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.610047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.610664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.610952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.611211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.611875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.613392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:14.616005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING keystone.server.flask.common [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 31 18:25:14.632989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.sql.core [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 18:25:14.632989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 31 18:25:14.632989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [parameters: {'user_id': 'eed93151c7374171a99ec00a2d389c47', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 31 18:25:14.632989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80202) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Jul 31 18:25:14.650556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 18:25:14.651609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application yield resource Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application next(self.gen) Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise self.value Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application session.commit() Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 18:25:14.652612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.session.flush() Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application compat.raise_( Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 18:25:14.654487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 18:25:14.656648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application conn.query(q) Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application result.read() Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 18:25:14.658695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [parameters: {'user_id': 'eed93151c7374171a99ec00a2d389c47', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:14.660743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 31 18:25:14.662717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:14.664857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 479/963] 10.222.0.49 () {60 vars in 1131 bytes} [Sun Jul 31 18:25:14 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:14.665762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:14.769688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c93fe0-0d20-4e25-b9fe-358e19b79892 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.777349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:14.804979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.805622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.805942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.806209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.813550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.814991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-970bb8aa-ae15-4e50-b7b5-6ad4828d098b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:14.826638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:14.828830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:14.829809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 485/964] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:14.834055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:14.945072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e12e-d3a9-43b3-b180-4d5730cc3eba None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:14.971500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:14.980432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:14.981140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:14.981415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:14.981683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:14.982441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:14.982734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:14.984385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d488393-32b0-497f-9b7a-bcafbf94d151 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:15.024949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 480/965] 10.222.0.49 () {56 vars in 1101 bytes} [Sun Jul 31 18:25:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:16.235895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458d508-db42-49b3-8dc4-6e727a1b62f4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.236384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458d508-db42-49b3-8dc4-6e727a1b62f4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.236384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458d508-db42-49b3-8dc4-6e727a1b62f4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.237992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 486/966] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:16.243101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.243523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.243597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.262646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:16.264129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:16.278066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23476-fed3-4d91-b811-3d412510ff17 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.278435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23476-fed3-4d91-b811-3d412510ff17 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.278713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d23476-fed3-4d91-b811-3d412510ff17 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.280269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 487/967] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:16.287991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.288533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.289075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.307307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:16.308499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:16.343269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 481/968] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:16.365975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.380816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 488/969] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:16.403352 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.468121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428cd6a-4d86-4cdd-8d30-80b4b29f71dc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:16.477268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:16.494704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:16.501428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8a43-a98d-4891-9b10-7db4a2753946 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:16.503119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebde2f3-f5cc-4f06-8255-5d77b9095458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:16.503739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebde2f3-f5cc-4f06-8255-5d77b9095458 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.504014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebde2f3-f5cc-4f06-8255-5d77b9095458 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.504285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebde2f3-f5cc-4f06-8255-5d77b9095458 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.505005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebde2f3-f5cc-4f06-8255-5d77b9095458 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:16.506761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebde2f3-f5cc-4f06-8255-5d77b9095458 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:16.521455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 482/970] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:16 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:16.522610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97222-8d2e-4967-a0b0-536523c9ca1a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.522897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97222-8d2e-4967-a0b0-536523c9ca1a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.523176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97222-8d2e-4967-a0b0-536523c9ca1a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.524396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 483/971] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:16.530403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.530878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.531000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.550080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:16.551465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:16.553390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7214639-f792-415d-9833-df07badd493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:16.554122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7214639-f792-415d-9833-df07badd493f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.554399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7214639-f792-415d-9833-df07badd493f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.554662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7214639-f792-415d-9833-df07badd493f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.569495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7214639-f792-415d-9833-df07badd493f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d7214639-f792-415d-9833-df07badd493f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:16.574666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:16.575904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:16.582562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 489/972] 10.222.0.49 () {56 vars in 1120 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:16.584181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7214639-f792-415d-9833-df07badd493f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.633378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 484/973] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:16.656226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.704641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7214639-f792-415d-9833-df07badd493f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:16.741146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:16.741689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.741966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.742238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.743075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:16.743375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:16.744871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:16.753638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 490/974] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:16.759835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.766027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:16.770381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd1ad-32e7-4f59-98b4-acdf157a2c10 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:16.777010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:16.811312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:16.811960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.812234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.812500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.813151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:16.814516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:16.823359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:16.824610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:16.825782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:16.829684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 485/975] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:16.830379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.871192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740accb-f52f-45e9-b6db-22ebfe73db34 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:16.906660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:16.907309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.907580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.907852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.919695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:16.921243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:16.936977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:16.937912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:16.943943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:16.943943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:16.943943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:16.943943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:16.944218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76a9d2-43d1-4528-959a-633adc859104 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:16.944305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 491/976] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:16.944714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:16.978219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:16.978837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:16.979119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:16.979378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:16.980042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:16.980346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:16.981720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:16.986710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 486/977] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:16.992692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.019507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.022296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.054862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d14d6b-352b-439a-8217-7dd4a801bc8d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.090507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.091135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.091408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.091678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.092363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:17.092714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.094122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.101974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782f069-6a6f-4ba1-920b-fd3248d2947b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.105903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 492/978] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:17.111529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.139442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.140292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.140713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.141225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.142175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:17.142619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.144185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.150007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 487/979] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:17.155653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.224790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61cf4d-4612-4ed5-99f7-d0814fe24249 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.255240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.257132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.266407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc913e-7c00-4914-9325-b7ac5fb62f03 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.267343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.268151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.268432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.269529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.278381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.280415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:17.295181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:17.296203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:17.297231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 493/980] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:17.302010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.303364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.304307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.304816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.305097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.307232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] RBAC: Adding query filter params (interface=admin, service_id=92d3d71cd8e24bc6a3c519b8647cf8dc, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:17.307795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.309571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.317558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 488/981] 10.222.0.49 () {56 vars in 1249 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/endpoints?service_id=92d3d71cd8e24bc6a3c519b8647cf8dc&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:17.322896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.410042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e54e0-9766-44eb-b1ff-3bf76e7e24e8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.433668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110059e1-a26d-488c-89bb-34e9a2aa2af9 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.444643 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.445349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.445680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.446015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.446881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:17.447243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.448733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.467121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 494/982] 10.222.0.49 () {56 vars in 1149 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:17.468617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.469322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.469658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.469988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.470700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.472110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.472600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.488802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 489/983] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:25:17.531707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.591606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa8efa-4e4c-4419-a806-19d2863bf07c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.630349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.631506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.631881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.632269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.709518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.711222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.758412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.819888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 495/984] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:25:17.821274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:17.842891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:17.926609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c5304-0984-46b8-9c74-9f7f218ed926 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:17.960589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:17.961263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:17.961518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:17.961809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:17.972579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:17.974161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:17.988485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:17.989578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:17.995065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:17.995065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:17.995065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:17.995065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:17.995065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 490/985] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:17.996713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:18.100788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779598-bddd-411f-8c2d-4b4d2ff04f26 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:18.135240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:18.135962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.136304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.136639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.137414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:18.137774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:18.139207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:18.155570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 496/986] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:18.157058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:18.237126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:18.269417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9787654-bce4-4b6d-bb14-0ef95b75508c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:18.303971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:18.304610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.304918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.305189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.312114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:18.313616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:18.325131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:18.325991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:18.331850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 491/987] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:18.333816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:18.338785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:18.440262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415bbc-83e7-4284-ab20-9b0061a09f38 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:18.474910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:18.475536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.475812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.476047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.476830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:18.477141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:18.478642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd8738-35d7-48a1-a2fa-afba8419c6c8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:18.496490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 497/988] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:18.497338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:18.519893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282d59-4e48-43cb-9d1d-140fa1c2597b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.520210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282d59-4e48-43cb-9d1d-140fa1c2597b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.520449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282d59-4e48-43cb-9d1d-140fa1c2597b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.521957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 498/989] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:18.526633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.526866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.527152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.545042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:18.546279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:18.605739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa579960-3e20-4db0-835a-7305691db3b6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:18.614135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:18.625361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 499/990] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:18.635719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:18.642446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:18.643069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.643343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.643610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.644393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] RBAC: Adding query filter params (role.id=f25d6c9b45554746b1730a94e6a72f2e, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=4242cff2780f478099110d0586ad67b3) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:18.644693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:18.646173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:18.648432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:18.652643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 492/991] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:18 2022] GET /identity/v3/role_assignments?user.id=4242cff2780f478099110d0586ad67b3&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=f25d6c9b45554746b1730a94e6a72f2e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:18.759734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00fb6d-e6b6-4795-bea0-94740c3b62f3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:18.820794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:18.820794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:18.820794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:18.820794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:18.821236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:18.821236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:18.831932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:18.833023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:18.833882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 500/992] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:18.838908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:18.973934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243498-ec40-45b0-a5c7-851b7037024c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:19.010937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:19.011893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.012365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.012865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.013762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:19.014275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:19.015918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:19.021834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 493/993] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:19.027567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:19.095620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:19.140850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b234e-3b8b-4148-a543-3a7052fa9cf8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:19.190593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:19.191490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.191798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.192066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.192854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:19.193275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:19.195312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:19.201069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 501/994] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:19.209439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:19.276679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:19.321036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631ab4a-3298-43d8-a74c-bf76ff413eb3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:19.354818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:19.355349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.355625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.355894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.356636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] RBAC: Adding query filter params (interface=public, service_id=1f91b4829cc24fdfa9f3ac744f5432e9, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:19.356966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:19.358414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd45cb-82b5-433c-8ff4-4b5f9f381a94 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:19.363945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 494/995] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:25:19 2022] GET /identity/v3/endpoints?service_id=1f91b4829cc24fdfa9f3ac744f5432e9&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:19.369522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:19.470329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc5d3f-c714-4fbc-804f-2e14b5b7f76e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:19.492448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:19.505442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:19.506435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.506713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.506979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.507619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:19.511847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:19.511964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166383e-2bb7-46c3-8327-c45a8756169e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.511964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166383e-2bb7-46c3-8327-c45a8756169e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.511964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166383e-2bb7-46c3-8327-c45a8756169e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.511964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 495/996] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:19.516696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.517216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.517687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.518194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 502/997] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:25:19 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:25:19.537884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:19.539147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:19.616613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 496/998] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:19.639419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:19.752022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f401ab-810e-4f2a-9b79-911a350dfb0b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:19.788106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:19.789074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.789558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.790005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.790823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:19.792401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:19.802744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:19.803850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 18:25:19.804692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:19.806024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 503/999] 10.222.0.49 () {56 vars in 1126 bytes} [Sun Jul 31 18:25:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:19.812693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:19.869132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:19.920569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d770573-17ca-4eab-ae6c-61a29126bd3d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:19.955737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:19.956350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:19.956625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:19.956926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:19.957604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:19.957890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:19.959247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:19.964929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 497/1000] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:19.970395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:20.025249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:20.077567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c563c-29a1-45b6-a846-df4adf3d8665 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:20.112445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:20.113074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:20.113349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:20.113612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:20.114287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:20.114586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:20.115952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf5f71-70db-4b50-bbaf-2894d275569e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:20.121456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 504/1001] 10.222.0.49 () {56 vars in 1122 bytes} [Sun Jul 31 18:25:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:20.127058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:20.237741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee945f90-1fcb-4680-afa9-aab31cf45db7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:20.254574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:20.278868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ced69-dcc4-4456-a2b2-53957a2932ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:20.279500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ced69-dcc4-4456-a2b2-53957a2932ce None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:20.279737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ced69-dcc4-4456-a2b2-53957a2932ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:20.280082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ced69-dcc4-4456-a2b2-53957a2932ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:20.280810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ced69-dcc4-4456-a2b2-53957a2932ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:20.282176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ced69-dcc4-4456-a2b2-53957a2932ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:20.307754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 498/1002] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:25:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:20.684915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dda795-94ac-4018-bc1f-f3e5a0649e00 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:20.685458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dda795-94ac-4018-bc1f-f3e5a0649e00 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:20.685458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dda795-94ac-4018-bc1f-f3e5a0649e00 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:20.686841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 505/1003] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:20.692083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:20.692364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:20.692630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:20.710843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:20.712008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:20.788592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 499/1004] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:20.810327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dda795-94ac-4018-bc1f-f3e5a0649e00 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:20.916508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dda795-94ac-4018-bc1f-f3e5a0649e00 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:20.930214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:20.952650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:20.953299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:20.953574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:20.953888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:20.954515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:20.955879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:20.965226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:20.966199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:20.967035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 506/1005] 10.222.0.49 () {56 vars in 1105 bytes} [Sun Jul 31 18:25:20 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:20.974496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:21.079198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672b8f8-4c25-4082-bc71-35f83ccae1fb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:21.080754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:21.117949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:21.118542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:21.118810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:21.119067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:21.119754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:21.120047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:21.121421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1efd3-8581-4c5a-9fb0-f7c8f4d91918 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:21.126622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 500/1006] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:21.132582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:21.242145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381526cb-c8ef-4ab0-8d6c-79450cf12199 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:21.277647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:21.278257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:21.278525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:21.278785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:21.279453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:21.279747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:21.281165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12fa49-fa4a-4bf2-9604-0a7c3bb7af19 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:21.285911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 507/1007] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:21.528533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4db5f8-2c9d-4a7e-b9c3-f4a7b62a300b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:21.529081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4db5f8-2c9d-4a7e-b9c3-f4a7b62a300b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:21.529081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4db5f8-2c9d-4a7e-b9c3-f4a7b62a300b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:21.530594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 501/1008] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:21.535701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:21.535989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:21.536233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:21.554854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:21.556041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:21.592993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:21.627044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 508/1009] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:21.650418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4db5f8-2c9d-4a7e-b9c3-f4a7b62a300b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:21.763213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4db5f8-2c9d-4a7e-b9c3-f4a7b62a300b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:21.797790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:21.798352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:21.798629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:21.798899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:21.799527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:21.800923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:21.810213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:21.811250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 18:25:21.812084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:21.816635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 502/1010] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:21.817530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:21.918151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d2987-16ea-44f2-bb7e-e4a85fb88594 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:21.953305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:21.953935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:21.954212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:21.954481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:21.955158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:21.955461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:21.956859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:21.962110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 509/1011] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:21.967833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:21.999720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:22.071412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e806285-97bd-4aaf-a835-9fbe8ba08263 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:22.106007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:22.106623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:22.106899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:22.107157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:22.107833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:22.108185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:22.109684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123320d2-f2e7-451c-b65b-105a12e9c3b8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:22.114967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 503/1012] 10.222.0.49 () {56 vars in 1112 bytes} [Sun Jul 31 18:25:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:22.120526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:22.157025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:22.232221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f2740-98db-4c26-a890-4b4e0529c34d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:22.266953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3b81d-c566-4a10-a270-cb54a2835cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:22.267591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3b81d-c566-4a10-a270-cb54a2835cb0 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:22.267870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3b81d-c566-4a10-a270-cb54a2835cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:22.268138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3b81d-c566-4a10-a270-cb54a2835cb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:22.268873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3b81d-c566-4a10-a270-cb54a2835cb0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:22.270235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3b81d-c566-4a10-a270-cb54a2835cb0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:22.293113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 510/1013] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:25:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:22.335354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516581cf-657c-45c3-9e28-66d9a1b931be None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:22.335564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516581cf-657c-45c3-9e28-66d9a1b931be None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:22.335806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516581cf-657c-45c3-9e28-66d9a1b931be None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:22.337423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 504/1014] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:22.343356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:22.343700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:22.343870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:22.363930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:22.365276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:22.450163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 511/1015] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:22.474658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516581cf-657c-45c3-9e28-66d9a1b931be None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:22.571760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:22.581340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516581cf-657c-45c3-9e28-66d9a1b931be None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:22.615693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363cf98-880e-4154-8d53-0e5b2788c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:22.616239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cf98-880e-4154-8d53-0e5b2788c214 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:22.616516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cf98-880e-4154-8d53-0e5b2788c214 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:22.616840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363cf98-880e-4154-8d53-0e5b2788c214 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:22.618085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363cf98-880e-4154-8d53-0e5b2788c214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:22.619504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363cf98-880e-4154-8d53-0e5b2788c214 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:22.621315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 505/1016] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:22.627210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:22.648826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:22.734071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef7da2-22cf-47be-9c8b-0b94ae8eacf1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:22.772057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13b20f-85a5-4167-9850-b6420ca85c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:22.772683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b20f-85a5-4167-9850-b6420ca85c70 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:22.772986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b20f-85a5-4167-9850-b6420ca85c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:22.773254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b20f-85a5-4167-9850-b6420ca85c70 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:22.773927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13b20f-85a5-4167-9850-b6420ca85c70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:22.775361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13b20f-85a5-4167-9850-b6420ca85c70 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:22.803439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 512/1017] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:22 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:23.273308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb9502-16e4-4bee-8f63-903bf6453c97 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:23.273652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb9502-16e4-4bee-8f63-903bf6453c97 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:23.273813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb9502-16e4-4bee-8f63-903bf6453c97 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:23.275402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 506/1018] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:23.280528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f39ca-f15f-4ac9-bddf-0d453b39e9ac None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:23.281034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f39ca-f15f-4ac9-bddf-0d453b39e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:23.281034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f39ca-f15f-4ac9-bddf-0d453b39e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:23.299664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-213f39ca-f15f-4ac9-bddf-0d453b39e9ac None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:23.301072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-213f39ca-f15f-4ac9-bddf-0d453b39e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:23.366915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:23.376922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 513/1019] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:23 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:23.395473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb9502-16e4-4bee-8f63-903bf6453c97 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:23.503474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb9502-16e4-4bee-8f63-903bf6453c97 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:23.541369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74131985-3d87-4718-bfae-2bd1a0aeaddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:23.541985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131985-3d87-4718-bfae-2bd1a0aeaddc None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:23.542261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131985-3d87-4718-bfae-2bd1a0aeaddc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:23.542531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131985-3d87-4718-bfae-2bd1a0aeaddc None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:23.543231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74131985-3d87-4718-bfae-2bd1a0aeaddc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:23.544605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74131985-3d87-4718-bfae-2bd1a0aeaddc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:23.568794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 507/1020] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:23 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:24.404072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78ec9a-ed9a-4059-9e33-4099800bb694 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:24.404614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78ec9a-ed9a-4059-9e33-4099800bb694 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:24.404614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78ec9a-ed9a-4059-9e33-4099800bb694 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:24.406004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 514/1021] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:24.411648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:24.412086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:24.412159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:24.430672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:24.431852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:24.450344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:24.509342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 508/1022] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:24.532501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78ec9a-ed9a-4059-9e33-4099800bb694 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:24.637887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78ec9a-ed9a-4059-9e33-4099800bb694 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:24.676214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ccbcd-2765-432c-b775-1265cc93ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:24.676897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ccbcd-2765-432c-b775-1265cc93ab57 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:24.677171 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ccbcd-2765-432c-b775-1265cc93ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:24.677459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ccbcd-2765-432c-b775-1265cc93ab57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:24.689790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ccbcd-2765-432c-b775-1265cc93ab57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1d3ccbcd-2765-432c-b775-1265cc93ab57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:24.695140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:24.696085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:24.703063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 515/1023] 10.222.0.49 () {56 vars in 1120 bytes} [Sun Jul 31 18:25:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:24.705498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:24.817263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2da0-ad18-457f-be19-1153d64d1123 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:24.828989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b984b-1d8d-4b65-994e-33da9e3e2137 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:24.829266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b984b-1d8d-4b65-994e-33da9e3e2137 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:24.830117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b984b-1d8d-4b65-994e-33da9e3e2137 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:24.831490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 516/1024] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:24.836382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:24.836638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:24.836949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:24.856569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:24.857215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:24.857336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:24.857495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:24.857759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:24.858513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:24.858609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:24.858815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:24.860179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:24.869146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 509/1025] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:25:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:24.876065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:24.889409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:24.926381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:24.939571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 517/1026] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:24.963345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:24.987219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d026d-e009-4873-89c8-c4d09e93dda7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.022732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.023369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.023639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.023903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.035537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.037019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:25.059057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:25.060016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:25.060960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:25.060960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:25.060960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:25.060960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:25.060960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 510/1027] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:25.064183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.079206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3987c5-8fae-4799-8866-17aaf4853c63 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.117465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.118127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.118405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.118664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.119819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.121283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.123331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 518/1028] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:25.128978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.137989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:25.179937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc4d61-17c9-4a3f-a1fd-0f0084cb5ba3 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.188888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:25.216613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.217326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.217603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.217870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.218559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:25.218857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.220220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.233435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 511/1029] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:25.239090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.246825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d68134-dd4e-4c68-a4ed-636e3f79196c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.282091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.282712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.282982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.283246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.283915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.285371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.311391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 519/1030] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:25:25 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:25.354266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b673-bb61-4b96-ae08-e3ca7c7f59ae None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.380282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:25.395292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.395952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.396223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.396483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.404752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.406270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c74497fd-8509-4b5c-84f8-e9f101720ab8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:25.419030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:25.420556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:25.421581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 512/1031] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:25.426536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.531389 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4103-ca42-4704-8e5e-d5de531ab26d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.545663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:25.550450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571746cf-efbb-4317-8709-cca034af51aa None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.550758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571746cf-efbb-4317-8709-cca034af51aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.551058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571746cf-efbb-4317-8709-cca034af51aa None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.552753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 513/1032] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:25.557903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.558180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.558451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.572391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.573052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.573325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.573588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.574379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:25.574676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.576104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.576567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:25.577813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:25.594672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 520/1033] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:25.600332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.652922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 514/1034] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:25.672627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.706392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387ccbe-f803-4ee4-99ae-adcc38e2329f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.740252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.740893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.741166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.741434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.781637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915d14-1a12-4b58-876b-c00b43b9ecc4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.782752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:25.796872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:25.812293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.813784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.819557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.820307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.820680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.821081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] PATH_INFO: `/v3/services/image` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.821711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:25.823068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:25.835017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:25.836085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:25.836967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:25.838088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 515/1035] 10.222.0.49 () {56 vars in 1105 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:25.843362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.918920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 521/1036] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:25:25.924606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:25.960413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbcc04-2cdc-46a9-9551-f56d6ec5ff57 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:25.997660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:25.998295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:25.998571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:25.998840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:25.999522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] RBAC: Adding query filter params (name=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:25.999823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.000204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:26.001420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.007676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 516/1037] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:26.013484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.038050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073cca3-5968-439e-9116-1445da5584c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.056282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:26.084486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.085827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.086372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.086875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.097544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.099009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:26.115918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:26.116858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:26.117781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:26.117781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:26.117781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:26.117781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:26.117781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 522/1038] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:26.122999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.127270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de924d88-9ded-44c1-8959-07050fa4bbcc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.162719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.163467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.163744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.164018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.164688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] RBAC: Adding query filter params (type=image) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:26.165013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.166374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.171683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 517/1039] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:26.177442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.242010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5609bd-2e84-4147-b934-833b5ea9c955 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.269346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:26.276709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d6289-e7dd-42c9-9727-f71edd9bd289 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.279080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:26.283123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.283781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.284443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.284729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.285448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:26.285748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.287476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.300046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 523/1040] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:26.305431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.313085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.313720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.313988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.314257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.315013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] RBAC: Adding query filter params (interface=public, service_id=b6ebfe64f15649a1bea8a91b22b612ab, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:26.315315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.316674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.322529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 518/1041] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/endpoints?service_id=b6ebfe64f15649a1bea8a91b22b612ab&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:26.328069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.409899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d32ba-c994-4d2b-b02e-3fa90a8f2fa4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.434558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd7741-046b-468e-a879-17b8116bdd96 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.448877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.449523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.449777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.450018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.457467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.458947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:26.471815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:26.472875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:26.473778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:26.474636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.474636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.474636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.474636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.474926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.475573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.478074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 524/1042] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:26.478777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.493209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 519/1043] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:25:26.531557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:26.593044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec248da9-14b5-4e3d-b16e-5139af97ab6b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.631711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.632332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.632602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.632890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.633638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:26.633932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.635350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff87be-c84a-427e-8273-f18dd5287d93 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.648413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 525/1044] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:26.654758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:26.691207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:26.762347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8279a2-89f9-4819-9517-4b62aefe2394 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:26.796797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:26.797434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:26.797702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:26.797961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:26.798745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] RBAC: Adding query filter params (role.id=f25d6c9b45554746b1730a94e6a72f2e, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=9958737194be4488a1e09e6e48557d75) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:26.799038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:26.800450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc574ede-925b-4890-9e9e-0861e2877340 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:26.806337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 520/1045] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:26 2022] GET /identity/v3/role_assignments?user.id=9958737194be4488a1e09e6e48557d75&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=f25d6c9b45554746b1730a94e6a72f2e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:27.350098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4357b-94a9-4c9d-910a-2b684b026bbd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:27.358409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4357b-94a9-4c9d-910a-2b684b026bbd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:27.358409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4357b-94a9-4c9d-910a-2b684b026bbd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:27.358409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 526/1046] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:27.360065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:27.360316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:27.360629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:27.380650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:27.382221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:27.459906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 521/1047] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:27.477191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4357b-94a9-4c9d-910a-2b684b026bbd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:27.583277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4357b-94a9-4c9d-910a-2b684b026bbd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:27.586793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:27.622905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:27.623596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:27.623868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:27.624131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:27.634989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:27.639899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:27.641212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:27.642117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:27.643297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 527/1048] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:27.647135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:27.675102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:27.749597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a245-3e1a-4361-b7be-1da5e6a689eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:27.784547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:27.785218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:27.785492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:27.785756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:27.786465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:27.786761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:27.788124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:27.796509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 522/1049] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:27.802876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:27.915381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d819890-343e-4f9d-9745-4ed1abd6fc04 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:27.949379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:27.950024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:27.950298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:27.950569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] PATH_INFO: `/v3/users/912b888a5eb14997b33c3bf08765bab8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:27.964045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=912b888a5eb14997b33c3bf08765bab8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:27.965525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:27.967420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 528/1050] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:27 2022] GET /identity/v3/users/912b888a5eb14997b33c3bf08765bab8 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:27.973133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:28.061002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:28.076345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610cf58-cace-4119-a508-1dccbc1b005a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:28.110179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:28.110803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.111080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.111347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.179133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:28.180578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:28.283897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:28.298827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 523/1051] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:25:28.304604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:28.330230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:28.409598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254a43-54ff-446c-8710-d9309f1b5cc7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:28.443480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f6d2b-7423-4ef9-9ff8-55c7d48686cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:28.444127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f6d2b-7423-4ef9-9ff8-55c7d48686cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.444408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f6d2b-7423-4ef9-9ff8-55c7d48686cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.444664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f6d2b-7423-4ef9-9ff8-55c7d48686cc None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.445865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f6d2b-7423-4ef9-9ff8-55c7d48686cc None admin] RBAC: Authorizing `identity:get_project(project_id=afef96908ca64b11b8e4f366c7161f34)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:28.447307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f6d2b-7423-4ef9-9ff8-55c7d48686cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:28.449193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 529/1052] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:28 2022] GET /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:28.455210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:28.474692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc48df-d273-4693-b45d-5f9b39762437 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.475083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc48df-d273-4693-b45d-5f9b39762437 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.475354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc48df-d273-4693-b45d-5f9b39762437 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.476726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 530/1053] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:28.481559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.482129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.482129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.500166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:28.501355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:28.573497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 531/1054] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:28.577000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b51d49-7089-4ea5-ba49-6896bca1e7c5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:28.595591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:28.611358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:28.611974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.612251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.612511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.613326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=afef96908ca64b11b8e4f366c7161f34, user.id=912b888a5eb14997b33c3bf08765bab8) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:28.613736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:28.615049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b02da1-52c2-4447-86df-85be8e67771d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:28.618578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:28.628802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 524/1055] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:28 2022] GET /identity/v3/role_assignments?user.id=912b888a5eb14997b33c3bf08765bab8&scope.project.id=afef96908ca64b11b8e4f366c7161f34&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:28.715103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04059e74-8f7e-4875-b05d-9cfc978b8077 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:28.752781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:28.753445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.753714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.753974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.754601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:28.756354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:28.762212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9b694-f9fb-4690-813e-9b7027634dab None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.762506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9b694-f9fb-4690-813e-9b7027634dab None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.762914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9b694-f9fb-4690-813e-9b7027634dab None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.764699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 525/1056] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:28.766278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:28.767287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:28.768702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 532/1057] 10.222.0.49 () {56 vars in 1105 bytes} [Sun Jul 31 18:25:28 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:28.774425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.774578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:28.774677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.775055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.795524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:28.796748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:28.850114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:28.864071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:28.871621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 526/1058] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:28.882856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a13416-26e4-4131-b2d0-a53c59d06aaf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:28.895228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:28.922445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:28.923075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:28.923341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:28.923603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:28.924271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] RBAC: Adding query filter params (name=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:28.924563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:28.925947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:28.936429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 533/1059] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:28.937450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.008569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a505ede-c4ef-493f-bd83-f2437086e9cb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.042582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6753b1-96ca-4a06-a520-9216c65b7bdc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.043882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.044716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.045043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.045412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.058705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:29.063728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:29.064666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:29.065686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:29.069825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 527/1060] 10.222.0.49 () {56 vars in 1120 bytes} [Sun Jul 31 18:25:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:29.071088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.082723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.083320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.083596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.083863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.084533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:29.084875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.086266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.096122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 534/1061] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:29.097189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.112866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:29.133710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:29.192902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5487fb2-34ec-4638-923f-1da216c866cb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.215710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d2d41-7b4b-4aa2-b36a-5eb54030c9b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.231567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.232194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.232467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.232793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.233511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:29.233868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.235256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.249711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 528/1062] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:25:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:29.251492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.259194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.259822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.260095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.260359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.261106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.262479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.293137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 535/1063] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:25:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:29.365493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3fbf-dbd0-4aa7-bfef-405df1ad7106 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.368012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:29.411236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.411882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.412152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.412415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.424172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.425661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:29.441335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:29.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:29.443230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:29.443230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:29.443230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:29.443230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:29.443230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 529/1064] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:29.448707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.519446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:29.555505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ad2f8-0621-485a-812e-0fa3d9fc5463 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.591476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.592106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.592406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.592675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.593395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:29.593695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.595069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.607598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 536/1065] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:29.613195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.719051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e037e-3f72-4a28-a160-a9bc416994ee None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.753632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.754272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.754544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.754808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.763572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.765111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:29.777951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:29.778927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:29.780522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 530/1066] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:25:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:29.784900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.890026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8c98-0d60-43ed-b034-7f3e365cb56e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:29.917323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:29.926594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:29.927214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:29.927482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:29.927741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:29.928530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:29.928857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:29.930278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:29.952198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 537/1067] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:25:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:29.953725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:29.969381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:30.056651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8974b8-23cd-49d5-83b4-3aaa46f5efab None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:30.090001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:30.090630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.090904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.091163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.101570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:30.102998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:30.117112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:30.118016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:30.124137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:30.124137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:30.124137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:30.124137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:30.124137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 531/1068] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:30.125671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:30.239074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cdc6b-362b-4d38-9a80-04fa5a922ece None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:30.274322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:30.274954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.275229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.275499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.276190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:30.276491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:30.277904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:30.289841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 538/1069] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:30.295420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:30.361907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:30.402403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b14be9-b062-4d2a-903f-1b86dcbcc897 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:30.439898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:30.440536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.440840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.441114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.447982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:30.449480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-54e8ba09-9ebc-4920-b032-c6752104de4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:30.461279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:30.462218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:30.463144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:30.465088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 532/1070] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:30.468317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:30.518940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:30.573777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73227a4-9b07-4ffa-95e4-0f621382e504 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:30.594933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9015f-85fd-4a5d-9879-28447f1deb4c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.595210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9015f-85fd-4a5d-9879-28447f1deb4c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.595496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9015f-85fd-4a5d-9879-28447f1deb4c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.597033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 533/1071] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:30.601585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.601865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.602141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.608727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:30.609375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.609654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.609918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.610658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:30.610957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:30.612376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:30.620927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:30.622113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:30.625586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 539/1072] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:30.631511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:30.696506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 534/1073] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:30.718110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:30.737357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422cbe8-9e60-434d-9084-eacdc73b5e8c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:30.749933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:30.773759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:30.786040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:30.786710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.786978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.787240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/9958737194be4488a1e09e6e48557d75/roles/f25d6c9b45554746b1730a94e6a72f2e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.789541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=9958737194be4488a1e09e6e48557d75, role_id=f25d6c9b45554746b1730a94e6a72f2e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:30.791236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:30.816871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 540/1074] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:25:30 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/9958737194be4488a1e09e6e48557d75/roles/f25d6c9b45554746b1730a94e6a72f2e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:25:30.824464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815db81d-bd69-4f58-9243-fafac5b73d90 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:30.863615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:30.864292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:30.864572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:30.864884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:30.877359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:30.882980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:30.884018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:30.890273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 535/1075] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:30.892983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:31.004188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93212-2d1c-436c-89c3-7dece02658ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:31.039224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:31.039762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.040041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.040309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.041046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:31.041344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:31.042805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:31.055750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 541/1076] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:31.057359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:31.162727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912cadb-935f-4d4b-b884-04a78d09b8ef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:31.177158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:31.198397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a06a26-efcf-4090-9669-254779743c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:31.199044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06a26-efcf-4090-9669-254779743c12 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.199322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06a26-efcf-4090-9669-254779743c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.199594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06a26-efcf-4090-9669-254779743c12 None admin] PATH_INFO: `/v3/users/912b888a5eb14997b33c3bf08765bab8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.200740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a06a26-efcf-4090-9669-254779743c12 None admin] RBAC: Authorizing `identity:get_user(user_id=912b888a5eb14997b33c3bf08765bab8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:31.202343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a06a26-efcf-4090-9669-254779743c12 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:31.204193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 536/1077] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:31 2022] GET /identity/v3/users/912b888a5eb14997b33c3bf08765bab8 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:31.209867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:31.243055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:31.313494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2397e2-8684-4a7d-8dfe-35bc02752584 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.313692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2397e2-8684-4a7d-8dfe-35bc02752584 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.313941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2397e2-8684-4a7d-8dfe-35bc02752584 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.315294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 537/1078] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:25:31.322663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.322946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.323218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.325582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b773ad-9af6-40a5-9dc1-b1ef70f0477b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:31.341822 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:31.343206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:31.369445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:31.370094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.370368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.370641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.371950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] RBAC: Authorizing `identity:get_project(project_id=afef96908ca64b11b8e4f366c7161f34)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:31.373563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:31.380334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 542/1079] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:31 2022] GET /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:31.381866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:31.434720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 538/1080] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:31.449412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:31.507404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:31.518598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fecf1-aa5d-46a6-a94c-4b4a8155ce0c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:31.560813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f0e25-6aeb-47cf-9560-ddf0c3b44600 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:31.575010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:31.581199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.581490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.581929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34/users/912b888a5eb14997b33c3bf08765bab8/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.583567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:31.584076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] RBAC: Authorizing `identity:create_grant(project_id=afef96908ca64b11b8e4f366c7161f34, user_id=912b888a5eb14997b33c3bf08765bab8, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:31.586797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:31.601261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 543/1081] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:25:31 2022] PUT /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34/users/912b888a5eb14997b33c3bf08765bab8/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:25:31.623689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:31.624445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.624676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.624971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.707126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:31.709168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:31.816724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 539/1082] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:25:31.822817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:31.930283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cdcc5-69e5-4f60-ac4c-4a05fbf3d696 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:31.964405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:31.965126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:31.965406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:31.965674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:31.976621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:31.978107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c3ebb627-f456-4a1f-aa61-3e4e1645878e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:31.992329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:31.993401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:31.994334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:31.994334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:31.994334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:31.994334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:31.995023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 544/1083] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:31.999743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:32.014885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:32.105833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d438d-594e-4f6f-98b1-edcf3d2e362c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:32.141659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:32.142346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:32.142626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:32.142899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:32.143579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:32.143879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:32.145294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb05c3a-5b14-43c6-9852-baa3e518c6c0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:32.156843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 540/1084] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:32.797787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e5ff-d57c-497d-a90f-dc71e5d8d35a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:32.798327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e5ff-d57c-497d-a90f-dc71e5d8d35a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:32.798327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e5ff-d57c-497d-a90f-dc71e5d8d35a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:32.799685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 545/1085] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:32.804956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:32.805353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:32.805503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:32.824386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:32.825630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:32.896025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 541/1086] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:32.919135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369e5ff-d57c-497d-a90f-dc71e5d8d35a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:32.932967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.027576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369e5ff-d57c-497d-a90f-dc71e5d8d35a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.062072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.062661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.062933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.063196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.074898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:33.079847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:33.080846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:33.081740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 546/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Sun Jul 31 18:25:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:33.087290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.089564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.188202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f25a08-a474-4524-83e8-85710e8f7ef5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.229873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.230511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.230787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.231056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.231776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:33.232073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:33.233478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:33.242540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 542/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:33.248294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.322065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.356513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a956a7-a7ef-4691-9a4e-d8bb84f55ccf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.391314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.391950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.392221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.392492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.403255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:33.404746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:33.419015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:33.420013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:33.421327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:33.421327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:33.421327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:33.421327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:33.424963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 547/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:33.425995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.489365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.534070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d349d0e-385d-4fc5-8322-85efceb66c1a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.568565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.569204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.569478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.569740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.570415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:33.570711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:33.572126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280b5bf-19e0-4ec5-b1df-5dafd2dd5ed9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:33.583977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 543/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:33.589792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.600000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ccea9-1298-4c59-9f89-3ddab5227bd2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.600288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ccea9-1298-4c59-9f89-3ddab5227bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.600560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ccea9-1298-4c59-9f89-3ddab5227bd2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.601916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 544/1091] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:33.606971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.607256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.607529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.625839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:33.627004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:33.691355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23036913-766f-48bb-a403-96a2816cd789 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.694024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 545/1092] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:33.711678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.716562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.727775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.728417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.728711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.729009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.736537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:33.738010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:25:33.749205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:33.750260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:33.751440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 548/1093] 10.222.0.49 () {56 vars in 1093 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:33.756435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.770582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.815672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ddef4-9480-4d38-b59c-e10e0fabf1f3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.848491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.849087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.849367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.849634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.854718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe96215-112a-4635-8143-5f611e5514f4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.859849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:33.864717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:33.865819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:33.866664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 546/1094] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:33.871922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.890353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:33.890979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:33.891253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:33.891520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:33.892308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:33.892642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:33.894185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:33.911283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 549/1095] 10.222.0.49 () {56 vars in 1133 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:33.916920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:33.956028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:33.973779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7417e-9009-4f14-bd7a-38cbf8337a18 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:33.988133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.008571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.009203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.009475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.009736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.010438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:34.010730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.012093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.019879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dd6fa-3fa7-4458-a626-ba9bb8d32779 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.020246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 547/1096] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:34.025981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.053131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.053746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.054048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.054319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.119946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.121371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.122875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb33c-b28a-4bc0-83ec-cfdca4a271ad None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.157700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.158329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.158603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.158871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] PATH_INFO: `/v3/users/912b888a5eb14997b33c3bf08765bab8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.159265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.160120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] RBAC: Authorizing `identity:get_user(user_id=912b888a5eb14997b33c3bf08765bab8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.161853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.163463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 548/1097] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/users/912b888a5eb14997b33c3bf08765bab8 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:34.169342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.213433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.219651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 550/1098] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:25:34.225204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.268717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9ece4-691f-470a-b802-97557c814b17 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.300713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.301321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.301590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.301852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.323174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6a997-8532-4d99-bb4c-d5b5f17ffe49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.357039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.357744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.358027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.358296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.364907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.367832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.368984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.369310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.370475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:25:34.385530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:34.386532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:34.387428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:25:34.387428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:25:34.387428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:25:34.387428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:34.387428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 551/1099] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:34.392315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.414528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.468745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 549/1100] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:25:34.474622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.491906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6f8a1-cbfa-43d9-9df4-6d6577147f85 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.525907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.526786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.527065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.527484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.528493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:34.528816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.531054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.545767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 552/1101] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:34.551274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.574122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4920ce9-4793-4b27-8820-5c10736edd11 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.606979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.608100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.608783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.609082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.609359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.610891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] RBAC: Authorizing `identity:get_project(project_id=afef96908ca64b11b8e4f366c7161f34)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.611972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.613690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 550/1102] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:34.619764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.635143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.657104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bc6b1-847f-46fa-a39c-dc5dc21753df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.689997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.690635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.690910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.691178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.697768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.699220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:25:34.711028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:34.712005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:34.712947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 553/1103] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:25:34.716464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d813c2-a572-49fd-929e-e5a62ba9603a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.718086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.749947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.750565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.750839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.751107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.751889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=afef96908ca64b11b8e4f366c7161f34, user.id=912b888a5eb14997b33c3bf08765bab8) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:34.752192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.753684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.760022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 551/1104] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/role_assignments?user.id=912b888a5eb14997b33c3bf08765bab8&scope.project.id=afef96908ca64b11b8e4f366c7161f34&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:34.828365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a152af-a09d-4754-b6bd-12ba53d292af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:34.840349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.869987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:34.870733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:34.870827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:34.871123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:34.871874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:34.872172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:34.873707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af096ba-4df1-4732-bd1b-7acde218171d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:34.887699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 554/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:34.894670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:34.975979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:34.998656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92643-9fbf-428e-bc1d-37abdb451d9a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:35.032460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:35.033102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:35.033373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:35.033636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:35.034419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] RBAC: Adding query filter params (role.id=f25d6c9b45554746b1730a94e6a72f2e, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=9958737194be4488a1e09e6e48557d75) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:35.034694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:35.036136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e45347-0cb2-431a-a7ec-2e54a10a6ef2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:35.041928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 552/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:34 2022] GET /identity/v3/role_assignments?user.id=9958737194be4488a1e09e6e48557d75&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=f25d6c9b45554746b1730a94e6a72f2e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:36.761099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f4a74-dbcf-4946-8d11-8d07d1aae77a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:36.761997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f4a74-dbcf-4946-8d11-8d07d1aae77a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:36.761997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f4a74-dbcf-4946-8d11-8d07d1aae77a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:36.764087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 555/1107] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:36.771936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:36.772382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:36.772494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:36.792305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:36.793499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:36.872575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 553/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:25:36.897424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f4a74-dbcf-4946-8d11-8d07d1aae77a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:37.005396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f4a74-dbcf-4946-8d11-8d07d1aae77a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:37.042170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ac0b9-894d-4f8c-b6c4-52915ae7635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:37.042735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ac0b9-894d-4f8c-b6c4-52915ae7635e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:37.043011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ac0b9-894d-4f8c-b6c4-52915ae7635e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:37.043277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ac0b9-894d-4f8c-b6c4-52915ae7635e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:37.044527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ac0b9-894d-4f8c-b6c4-52915ae7635e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:37.046094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ac0b9-894d-4f8c-b6c4-52915ae7635e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:37.047690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 556/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:37.056242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:37.126868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:37.160786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadda6cc-3e2e-49c4-a1e0-3a68eaabfec0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:37.196316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df054b-32d8-41cb-9bed-541e2c73ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:37.196950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df054b-32d8-41cb-9bed-541e2c73ef62 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:37.197222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df054b-32d8-41cb-9bed-541e2c73ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:37.197488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df054b-32d8-41cb-9bed-541e2c73ef62 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:37.198161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df054b-32d8-41cb-9bed-541e2c73ef62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:37.199571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df054b-32d8-41cb-9bed-541e2c73ef62 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:37.225534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 554/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:25:37 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:25:39.222070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61381-7251-49fc-9604-d2fde1d26ac9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:39.222070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61381-7251-49fc-9604-d2fde1d26ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:39.222070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61381-7251-49fc-9604-d2fde1d26ac9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:39.223293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 557/1111] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:39.231760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:39.232482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:39.232644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:39.257157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:39.258352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:39.325866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 555/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:39.364069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61381-7251-49fc-9604-d2fde1d26ac9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:39.411368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:39.477752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61381-7251-49fc-9604-d2fde1d26ac9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:39.514769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:39.515398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:39.515673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:39.515938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] PATH_INFO: `/v3/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:39.517429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] RBAC: Authorizing `identity:get_role(role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:39.518811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:39.520156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 558/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:39 2022] GET /identity/v3/roles/c31497969e7944d1ae71b619ace9c773 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:39.529498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:39.592263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:39.640985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e883-e451-4ac4-9abc-aa2b5375b8ac None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:39.676402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:39.677058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:39.677336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:39.677603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] PATH_INFO: `/v3/users/26abdb7985a34772ae5035596ec93fa4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:39.691329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] RBAC: Authorizing `identity:get_user(user_id=26abdb7985a34772ae5035596ec93fa4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:39.692796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:39.694395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 556/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:39 2022] GET /identity/v3/users/26abdb7985a34772ae5035596ec93fa4 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:39.702830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:39.807051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c7202-6af6-4f51-9416-91dec95e4776 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:39.817597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:39.848209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:39.848819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:39.849116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:39.849383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:39.920029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:39.921500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:40.021481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 559/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:25:40.028380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:40.145579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2249f-9498-4446-a4c5-04643284660a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:40.159387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:40.184655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e489e08-b4e1-4298-83bb-a4704836bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:40.185307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e489e08-b4e1-4298-83bb-a4704836bc24 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:40.185578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e489e08-b4e1-4298-83bb-a4704836bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:40.185842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e489e08-b4e1-4298-83bb-a4704836bc24 None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:40.186979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e489e08-b4e1-4298-83bb-a4704836bc24 None admin] RBAC: Authorizing `identity:get_project(project_id=afef96908ca64b11b8e4f366c7161f34)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:40.188404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e489e08-b4e1-4298-83bb-a4704836bc24 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:40.190074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 557/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:40 2022] GET /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:40.196494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:40.204716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:40.306252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca644488-0a4d-41a8-8c57-6839777bc0cf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:40.348954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:40.349556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:40.349828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:40.350085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:40.350861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] RBAC: Adding query filter params (role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=afef96908ca64b11b8e4f366c7161f34, user.id=26abdb7985a34772ae5035596ec93fa4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:40.351153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:40.352590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e30e1-208a-47e9-958b-90ebc6d6d69a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:40.358142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 560/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:40 2022] GET /identity/v3/role_assignments?user.id=26abdb7985a34772ae5035596ec93fa4&scope.project.id=afef96908ca64b11b8e4f366c7161f34&role.id=c31497969e7944d1ae71b619ace9c773 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:25:42.294875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7aed9-33db-4560-b9c9-cca61259ea53 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:42.295741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7aed9-33db-4560-b9c9-cca61259ea53 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:42.295854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7aed9-33db-4560-b9c9-cca61259ea53 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:42.298098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 558/1118] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:42.307016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:42.307735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:42.307872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:42.334107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:42.335300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:42.400366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:42.405542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 561/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:42.428265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7aed9-33db-4560-b9c9-cca61259ea53 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:42.533475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7aed9-33db-4560-b9c9-cca61259ea53 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:42.568579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:42.569237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:42.569507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:42.569779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] PATH_INFO: `/v3/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:42.571337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] RBAC: Authorizing `identity:get_role(role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:42.572706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:42.574056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 559/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:42 2022] GET /identity/v3/roles/c31497969e7944d1ae71b619ace9c773 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:42.580074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:42.686511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df7ea5-e3ee-4828-b9c6-ccd5de7cade4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:42.720915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:42.721544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:42.721817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:42.722078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] PATH_INFO: `/v3/users/26abdb7985a34772ae5035596ec93fa4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:42.723255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] RBAC: Authorizing `identity:get_user(user_id=26abdb7985a34772ae5035596ec93fa4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:42.724685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:42.726330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 562/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:42 2022] GET /identity/v3/users/26abdb7985a34772ae5035596ec93fa4 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:42.735603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:42.792644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:42.841915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c83b9a-243c-499b-93b7-a25599cf0ec7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:42.876124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5733966-3f1c-4bd2-beff-65f8ab58f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:42.876746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5733966-3f1c-4bd2-beff-65f8ab58f85f None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:42.877049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5733966-3f1c-4bd2-beff-65f8ab58f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:42.877316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5733966-3f1c-4bd2-beff-65f8ab58f85f None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:42.878514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5733966-3f1c-4bd2-beff-65f8ab58f85f None admin] RBAC: Authorizing `identity:get_project(project_id=afef96908ca64b11b8e4f366c7161f34)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:42.879945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5733966-3f1c-4bd2-beff-65f8ab58f85f None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:42.882010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 560/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:42 2022] GET /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:42.888798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:42.967745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:43.005101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933a235-1260-4c59-b870-6f64375ba2dd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:43.041745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8922a3-fc21-458f-a6c6-0a25595073da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:43.042403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8922a3-fc21-458f-a6c6-0a25595073da None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:43.042741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8922a3-fc21-458f-a6c6-0a25595073da None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:43.042911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8922a3-fc21-458f-a6c6-0a25595073da None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34/users/26abdb7985a34772ae5035596ec93fa4/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:43.045582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8922a3-fc21-458f-a6c6-0a25595073da None admin] RBAC: Authorizing `identity:create_grant(project_id=afef96908ca64b11b8e4f366c7161f34, user_id=26abdb7985a34772ae5035596ec93fa4, role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:43.047034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8922a3-fc21-458f-a6c6-0a25595073da None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:43.067040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 563/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:25:42 2022] PUT /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34/users/26abdb7985a34772ae5035596ec93fa4/roles/c31497969e7944d1ae71b619ace9c773 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:25:45.049918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b19242-8439-4834-b799-b6acc258ee33 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:45.050835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b19242-8439-4834-b799-b6acc258ee33 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:45.050835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b19242-8439-4834-b799-b6acc258ee33 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:45.052973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 561/1124] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:45.059220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:45.059936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:45.060109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:45.080359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:45.081575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:45.164858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 564/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:45.196993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b19242-8439-4834-b799-b6acc258ee33 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:45.321327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b19242-8439-4834-b799-b6acc258ee33 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:45.338580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:45.358315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:45.358951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:45.359227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:45.359492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] PATH_INFO: `/v3/roles/c31497969e7944d1ae71b619ace9c773` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:45.361000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] RBAC: Authorizing `identity:get_role(role_id=c31497969e7944d1ae71b619ace9c773)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:45.362363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:45.363702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 562/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:45 2022] GET /identity/v3/roles/c31497969e7944d1ae71b619ace9c773 => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:45.369879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:45.428780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:45.472135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fe5ec-cd3a-465d-a11b-aa1b23e4eea5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:45.517986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:45.518622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:45.518902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:45.519167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] PATH_INFO: `/v3/users/26abdb7985a34772ae5035596ec93fa4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:45.520339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=26abdb7985a34772ae5035596ec93fa4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:45.521820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:45.523408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 565/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:45 2022] GET /identity/v3/users/26abdb7985a34772ae5035596ec93fa4 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:45.529555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:45.634861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f4184-91ed-4854-821d-62dbf1c7c06a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:45.670429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:45.671025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:45.671299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:45.671569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:45.729090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:45.743073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:45.744511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:45.849758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 563/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:25:45.855819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:45.960445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42bb51-2d87-44a3-a769-5f1ee0495ee2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:45.986092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:45.995693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fbf2d-3db4-4392-8acd-abfa347b38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:45.996314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fbf2d-3db4-4392-8acd-abfa347b38aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:45.996584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fbf2d-3db4-4392-8acd-abfa347b38aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:45.996880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fbf2d-3db4-4392-8acd-abfa347b38aa None admin] PATH_INFO: `/v3/projects/afef96908ca64b11b8e4f366c7161f34` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:45.998159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fbf2d-3db4-4392-8acd-abfa347b38aa None admin] RBAC: Authorizing `identity:get_project(project_id=afef96908ca64b11b8e4f366c7161f34)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:45.999586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fbf2d-3db4-4392-8acd-abfa347b38aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:46.001242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 566/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:45 2022] GET /identity/v3/projects/afef96908ca64b11b8e4f366c7161f34 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:46.011274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:46.100244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:46.118010 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a8294-a060-4219-86b3-9446a344a336 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:46.155416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:46.156024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:46.156298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:46.156560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:46.157377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] RBAC: Adding query filter params (role.id=c31497969e7944d1ae71b619ace9c773, scope.project.id=afef96908ca64b11b8e4f366c7161f34, user.id=26abdb7985a34772ae5035596ec93fa4) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:46.157678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:46.159121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900722e-8bee-47e8-8abc-ce7fdd5139a6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:46.165031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 564/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:46 2022] GET /identity/v3/role_assignments?user.id=26abdb7985a34772ae5035596ec93fa4&scope.project.id=afef96908ca64b11b8e4f366c7161f34&role.id=c31497969e7944d1ae71b619ace9c773 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:48.128849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81143670-76ce-45d5-8bf0-1937a6e9874f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:48.129446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81143670-76ce-45d5-8bf0-1937a6e9874f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:48.129446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81143670-76ce-45d5-8bf0-1937a6e9874f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:48.130727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 567/1131] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:48.137307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:48.138034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:48.138226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:48.162838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:48.164022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:48.237889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 565/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:48.261075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81143670-76ce-45d5-8bf0-1937a6e9874f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:48.364005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81143670-76ce-45d5-8bf0-1937a6e9874f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:48.405353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711539d1-bce8-4101-b75b-17d7aee3417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:48.406308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711539d1-bce8-4101-b75b-17d7aee3417b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:48.406692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711539d1-bce8-4101-b75b-17d7aee3417b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:48.407120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711539d1-bce8-4101-b75b-17d7aee3417b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:48.408942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711539d1-bce8-4101-b75b-17d7aee3417b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:48.411247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711539d1-bce8-4101-b75b-17d7aee3417b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:48.413750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 568/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:48.419755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:48.504492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:48.531190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bad8c8-efc7-44c8-98c4-60d83c7f6f7a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:48.570952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658f0f3-d82c-4cff-88b2-c573c2981ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:48.571557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f0f3-d82c-4cff-88b2-c573c2981ee9 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:48.571827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f0f3-d82c-4cff-88b2-c573c2981ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:48.572091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f0f3-d82c-4cff-88b2-c573c2981ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:48.572796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658f0f3-d82c-4cff-88b2-c573c2981ee9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:48.574217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658f0f3-d82c-4cff-88b2-c573c2981ee9 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:48.602412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 566/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:25:48 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:50.614115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e29414-1942-4405-9395-ed4ed54aae57 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:50.615000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e29414-1942-4405-9395-ed4ed54aae57 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:50.615000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e29414-1942-4405-9395-ed4ed54aae57 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:50.617175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 569/1135] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:50.625900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:50.626314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:50.626510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:50.650247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:50.651443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:50.721600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 567/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:50.749873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e29414-1942-4405-9395-ed4ed54aae57 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:50.819555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:50.864431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e29414-1942-4405-9395-ed4ed54aae57 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:50.902347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce85af-4160-4211-9f5a-2e9e71cb5714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:50.903635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce85af-4160-4211-9f5a-2e9e71cb5714 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:50.903993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce85af-4160-4211-9f5a-2e9e71cb5714 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:50.904576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce85af-4160-4211-9f5a-2e9e71cb5714 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:50.906303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce85af-4160-4211-9f5a-2e9e71cb5714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:50.908222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce85af-4160-4211-9f5a-2e9e71cb5714 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:50.910327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 570/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:25:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:50.916299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:50.979735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:51.029106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ceb613-c2f8-4207-ae27-65dfa733baad None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:51.067872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b69e2-7e2c-4ab6-92d8-e44a91aa6600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:51.068486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b69e2-7e2c-4ab6-92d8-e44a91aa6600 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:51.068791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b69e2-7e2c-4ab6-92d8-e44a91aa6600 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:51.069061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b69e2-7e2c-4ab6-92d8-e44a91aa6600 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:51.069744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b69e2-7e2c-4ab6-92d8-e44a91aa6600 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:51.071172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b69e2-7e2c-4ab6-92d8-e44a91aa6600 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:51.095922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 568/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:25:50 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:25:53.138962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dbd20-2c62-4948-bfca-b3f7345f6a04 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:53.139864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dbd20-2c62-4948-bfca-b3f7345f6a04 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:53.139864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dbd20-2c62-4948-bfca-b3f7345f6a04 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:53.141974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 571/1139] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:53.148124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:53.148887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:53.149085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:53.170052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:53.171251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:53.250138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 569/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:53.274375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dbd20-2c62-4948-bfca-b3f7345f6a04 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:53.401214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dbd20-2c62-4948-bfca-b3f7345f6a04 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:53.437864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:53.444909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:53.445548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:53.445825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:53.446094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:53.456701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:53.461645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:53.462730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:53.463783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:53.464911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 572/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:53.469257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:53.521267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:53.573106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504243-1d50-4cc6-8be1-f3e3855d6d7d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:53.611560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50918d22-d958-4598-ada9-372165278c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:53.612175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50918d22-d958-4598-ada9-372165278c21 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:53.612444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50918d22-d958-4598-ada9-372165278c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:53.612707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50918d22-d958-4598-ada9-372165278c21 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:53.613451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50918d22-d958-4598-ada9-372165278c21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:53.613753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50918d22-d958-4598-ada9-372165278c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:53.615109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50918d22-d958-4598-ada9-372165278c21 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:53.623766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 570/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:53.629833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:53.730404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176fb2e-0bc5-4e7a-b6b3-754fbac1eaf3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:53.765941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:53.766550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:53.766815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:53.767075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] PATH_INFO: `/v3/users/b2755b80ea4945debda5ab95e1a825d5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:53.780510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] RBAC: Authorizing `identity:get_user(user_id=b2755b80ea4945debda5ab95e1a825d5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:53.781974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:53.783541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 573/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:53 2022] GET /identity/v3/users/b2755b80ea4945debda5ab95e1a825d5 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:53.792828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50918d22-d958-4598-ada9-372165278c21 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:53.904623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50918d22-d958-4598-ada9-372165278c21 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:53.916193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:53.942546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:53.943145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:53.943417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:53.943682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:54.011771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:54.013155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:54.117527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 571/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:25:54.123868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:54.179148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:54.245993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e1bf2-4885-4d83-b673-600d2f4e44cf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:54.281008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c800018-3ccf-4f3f-9aa2-841fe12404d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:54.281572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c800018-3ccf-4f3f-9aa2-841fe12404d9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:54.281840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c800018-3ccf-4f3f-9aa2-841fe12404d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:54.282102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c800018-3ccf-4f3f-9aa2-841fe12404d9 None admin] PATH_INFO: `/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:54.283260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c800018-3ccf-4f3f-9aa2-841fe12404d9 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2b247c97ea4389bf5ff98ee3ea33b9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:54.284685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c800018-3ccf-4f3f-9aa2-841fe12404d9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:54.286364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 574/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:54 2022] GET /identity/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:54.296642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:54.308943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:54.402554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb69aa-49b9-40e2-af06-41b88c28c05d None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:54.437622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:54.438216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:54.438486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:54.438750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:54.439523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=1e2b247c97ea4389bf5ff98ee3ea33b9, user.id=b2755b80ea4945debda5ab95e1a825d5) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:54.439817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:54.441264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164283b-f63f-46bc-b7ca-cc6d31d4de55 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:54.446733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 572/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:25:54 2022] GET /identity/v3/role_assignments?user.id=b2755b80ea4945debda5ab95e1a825d5&scope.project.id=1e2b247c97ea4389bf5ff98ee3ea33b9&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:56.393961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2617e8-bb5c-450b-98a7-fe9362c71e4d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:56.394892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2617e8-bb5c-450b-98a7-fe9362c71e4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:56.394892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2617e8-bb5c-450b-98a7-fe9362c71e4d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:56.397017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 575/1147] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:56.403267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:56.403987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:56.404321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:56.422989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:56.424181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:56.493406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 573/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:56.517273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2617e8-bb5c-450b-98a7-fe9362c71e4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:56.623867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2617e8-bb5c-450b-98a7-fe9362c71e4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:56.640470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:56.660720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:56.661379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:56.661646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:56.661908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:56.672835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:56.677736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:56.678811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:25:56.679677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 576/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:56.685014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:56.688216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:56.790618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623513ce-994e-4315-99d9-bcf16a2e7bbf None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:56.825291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:56.825978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:56.826251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:56.826522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:56.827222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:56.827512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:56.828890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:56.837059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 574/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:56.845771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:56.953569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a46d35-4dba-4089-8983-cbbab2a2ed94 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:56.988440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:56.989086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:56.989360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:56.989629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] PATH_INFO: `/v3/users/b2755b80ea4945debda5ab95e1a825d5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:56.990905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] RBAC: Authorizing `identity:get_user(user_id=b2755b80ea4945debda5ab95e1a825d5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:56.992336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:56.993944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 577/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:56 2022] GET /identity/v3/users/b2755b80ea4945debda5ab95e1a825d5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:56.999773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:57.077631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:57.114207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a6bac-62c1-4918-81bd-1a10c8d915a7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:57.149046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f404e73-c5c9-489d-bdc2-ad99e2179469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:57.149669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404e73-c5c9-489d-bdc2-ad99e2179469 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:57.149940 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404e73-c5c9-489d-bdc2-ad99e2179469 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:57.150206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404e73-c5c9-489d-bdc2-ad99e2179469 None admin] PATH_INFO: `/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:57.151467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404e73-c5c9-489d-bdc2-ad99e2179469 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2b247c97ea4389bf5ff98ee3ea33b9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:57.152913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404e73-c5c9-489d-bdc2-ad99e2179469 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:57.154649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 575/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:25:56 2022] GET /identity/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:57.160927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:57.211358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:57.268386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ab8c8-eaf2-4b31-a63d-e0423f1c76e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:57.308195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ab7b6-e7b6-455c-91cc-9355af58ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:57.308885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ab7b6-e7b6-455c-91cc-9355af58ddf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:57.309236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ab7b6-e7b6-455c-91cc-9355af58ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:57.309510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ab7b6-e7b6-455c-91cc-9355af58ddf9 None admin] PATH_INFO: `/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9/users/b2755b80ea4945debda5ab95e1a825d5/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:57.311577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ab7b6-e7b6-455c-91cc-9355af58ddf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2b247c97ea4389bf5ff98ee3ea33b9, user_id=b2755b80ea4945debda5ab95e1a825d5, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:57.313055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ab7b6-e7b6-455c-91cc-9355af58ddf9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:57.329232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 578/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:25:57 2022] PUT /identity/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9/users/b2755b80ea4945debda5ab95e1a825d5/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:25:59.294848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f81e1-8f3f-4c6a-8eb1-4ab5ad11d235 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:59.295680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f81e1-8f3f-4c6a-8eb1-4ab5ad11d235 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:59.295680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f81e1-8f3f-4c6a-8eb1-4ab5ad11d235 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:59.298046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 576/1154] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:25:59.307483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:59.308177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:59.308359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:59.329646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:25:59.330834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:25:59.412936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 579/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:25:59.436113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f81e1-8f3f-4c6a-8eb1-4ab5ad11d235 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:59.557209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f81e1-8f3f-4c6a-8eb1-4ab5ad11d235 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:59.592397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:59.593041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:59.593313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:59.593578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:59.595768 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:59.608414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:25:59.613455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:25:59.614454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:25:59.615370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 577/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:25:59.620947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:59.657429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:25:59.726462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c1005-beba-4dc9-a9a2-906f0f93cdfc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:59.762897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:59.763502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:59.763834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:59.764105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:59.764809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:25:59.765132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:59.766507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:59.774555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 580/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:59.781218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:25:59.890225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d3be7-d038-4376-ae53-bca5ebb9c9f4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:25:59.926664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:25:59.927310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:25:59.927582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:25:59.927846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] PATH_INFO: `/v3/users/b2755b80ea4945debda5ab95e1a825d5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:25:59.929175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] RBAC: Authorizing `identity:get_user(user_id=b2755b80ea4945debda5ab95e1a825d5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:25:59.930615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:25:59.932222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 578/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:25:59 2022] GET /identity/v3/users/b2755b80ea4945debda5ab95e1a825d5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:25:59.938256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:00.036528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:00.043201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ebda3-dcdf-4cce-947b-21bda126b92b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:00.079215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:00.079822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:00.080097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:00.080358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:00.149339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:00.150709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:00.245584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:00.253366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 581/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:25:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:26:00.259735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:00.323411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:00.368001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cb1db-2ee4-4fbe-8fbe-b42f89ea5a1c None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:00.411544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a94a64-eb07-4657-86d2-4f9e32250831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:00.412081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a94a64-eb07-4657-86d2-4f9e32250831 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:00.412350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a94a64-eb07-4657-86d2-4f9e32250831 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:00.412616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a94a64-eb07-4657-86d2-4f9e32250831 None admin] PATH_INFO: `/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:00.413908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a94a64-eb07-4657-86d2-4f9e32250831 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2b247c97ea4389bf5ff98ee3ea33b9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:00.415341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a94a64-eb07-4657-86d2-4f9e32250831 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:00.417074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 579/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:26:00 2022] GET /identity/v3/projects/1e2b247c97ea4389bf5ff98ee3ea33b9 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:00.424696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:00.527834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400664b-5657-476a-ae52-e90c7867a476 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:00.562597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:00.563198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:00.563537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:00.563806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:00.564590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=1e2b247c97ea4389bf5ff98ee3ea33b9, user.id=b2755b80ea4945debda5ab95e1a825d5) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:00.564925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:00.566349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58856369-5435-44db-a421-fd863129b9a3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:00.572149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 582/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:26:00 2022] GET /identity/v3/role_assignments?user.id=b2755b80ea4945debda5ab95e1a825d5&scope.project.id=1e2b247c97ea4389bf5ff98ee3ea33b9&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:02.563714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2a59a-16c5-46f8-8379-fc055c41360d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:02.564263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2a59a-16c5-46f8-8379-fc055c41360d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:02.564263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2a59a-16c5-46f8-8379-fc055c41360d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:02.565557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 580/1162] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:02.571845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:02.572568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:02.572738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:02.594251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:02.595438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:02.667162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:02.680122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 583/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:26:02.701973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2a59a-16c5-46f8-8379-fc055c41360d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:02.810475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2a59a-16c5-46f8-8379-fc055c41360d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:02.839517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:02.846233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:02.846837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:02.847110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:02.847379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:02.916689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:02.918211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:03.023469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 581/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:26:03.030421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:03.161553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398e3b-2a4a-49ed-beda-e3be9813b4b8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:03.196173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:03.196810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:03.197098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:03.197360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:03.204272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:03.205750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-25dbe547-8c97-4801-821b-fdd5ce1dfd28 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:26:03.218145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:26:03.219231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:26:03.220237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 584/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:26:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:26:03.226209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:03.260637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:03.335167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d6be1-c63c-4e28-b40e-9ba320ff84c6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:03.370322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:03.370913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:03.371186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:03.371450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:03.372123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:03.372410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:03.373821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe78b1e-bc66-424d-b405-3c2a158af68e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:03.381293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 582/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Sun Jul 31 18:26:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:05.346365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5f31-20f2-4b1a-8187-ebcfdaa94e75 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:05.346939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5f31-20f2-4b1a-8187-ebcfdaa94e75 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:05.346939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5f31-20f2-4b1a-8187-ebcfdaa94e75 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:05.348254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 585/1167] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:05.354589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c2bb-6a52-41b8-af64-b0a967cc2a8e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:05.355309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c2bb-6a52-41b8-af64-b0a967cc2a8e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:05.355455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5c2bb-6a52-41b8-af64-b0a967cc2a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:05.380527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-49a5c2bb-6a52-41b8-af64-b0a967cc2a8e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:05.382125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-49a5c2bb-6a52-41b8-af64-b0a967cc2a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:05.455409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 583/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:26:05.478043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5f31-20f2-4b1a-8187-ebcfdaa94e75 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:05.536630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:05.598874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5f31-20f2-4b1a-8187-ebcfdaa94e75 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:05.632692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78dcaa-07fd-43dd-a8e3-69087ed4cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:05.633322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78dcaa-07fd-43dd-a8e3-69087ed4cb32 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:05.633594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78dcaa-07fd-43dd-a8e3-69087ed4cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:05.633858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78dcaa-07fd-43dd-a8e3-69087ed4cb32 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:05.634566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78dcaa-07fd-43dd-a8e3-69087ed4cb32 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:05.635929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78dcaa-07fd-43dd-a8e3-69087ed4cb32 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:05.657249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 586/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Sun Jul 31 18:26:05 2022] POST /identity/v3/domains => generated 267 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:26:07.644737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e6967-dd5e-4749-8cf2-2136c2ff1397 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:07.645651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e6967-dd5e-4749-8cf2-2136c2ff1397 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:07.645651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e6967-dd5e-4749-8cf2-2136c2ff1397 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:07.647784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 584/1170] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:07.655618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:07.656052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:07.656172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:07.675831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:07.677010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:07.744568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 587/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 18:26:07.769575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e6967-dd5e-4749-8cf2-2136c2ff1397 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:07.800140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:07.885652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e6967-dd5e-4749-8cf2-2136c2ff1397 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:07.928314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d02d4-7f2a-4f7e-8415-fe27174f346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:07.928909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d02d4-7f2a-4f7e-8415-fe27174f346b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:07.929215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d02d4-7f2a-4f7e-8415-fe27174f346b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:07.929453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d02d4-7f2a-4f7e-8415-fe27174f346b None admin] PATH_INFO: `/v3/domains/4424a3ffac3445ec82421136be7ef026` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:07.940095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d02d4-7f2a-4f7e-8415-fe27174f346b None admin] RBAC: Authorizing `identity:get_domain(domain_id=4424a3ffac3445ec82421136be7ef026)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:07.941582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d02d4-7f2a-4f7e-8415-fe27174f346b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:07.943184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 585/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Sun Jul 31 18:26:07 2022] GET /identity/v3/domains/4424a3ffac3445ec82421136be7ef026 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:07.949804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:07.999599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:08.061242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42896030-c5c7-4c95-94d6-9fb19a8d809d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:08.098204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870db450-6446-4901-929e-1f3719ef7099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:08.098812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870db450-6446-4901-929e-1f3719ef7099 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:08.099083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870db450-6446-4901-929e-1f3719ef7099 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:08.099347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870db450-6446-4901-929e-1f3719ef7099 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:08.100018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870db450-6446-4901-929e-1f3719ef7099 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:08.101492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870db450-6446-4901-929e-1f3719ef7099 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:08.132896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 588/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:26:07 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:26:10.132618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41436-88bc-4f02-975d-7df7c5a10b3c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:10.133547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41436-88bc-4f02-975d-7df7c5a10b3c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:10.133547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41436-88bc-4f02-975d-7df7c5a10b3c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:10.135730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 586/1174] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:10.141242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:10.141659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:10.141777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:10.162264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:10.163443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:10.245681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 589/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:26:10.271686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41436-88bc-4f02-975d-7df7c5a10b3c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:10.365308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:10.384838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41436-88bc-4f02-975d-7df7c5a10b3c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:10.423596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfd55e-81ab-48d4-9ef8-8addc70fd2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:10.424223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfd55e-81ab-48d4-9ef8-8addc70fd2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:10.424496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfd55e-81ab-48d4-9ef8-8addc70fd2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:10.424814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfd55e-81ab-48d4-9ef8-8addc70fd2c6 None admin] PATH_INFO: `/v3/domains/4424a3ffac3445ec82421136be7ef026` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:10.426118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfd55e-81ab-48d4-9ef8-8addc70fd2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4424a3ffac3445ec82421136be7ef026)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:10.427545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfd55e-81ab-48d4-9ef8-8addc70fd2c6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:10.429202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 587/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Sun Jul 31 18:26:10 2022] GET /identity/v3/domains/4424a3ffac3445ec82421136be7ef026 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:10.437929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:10.464408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:10.547016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cd328-84ea-4f09-a062-894b26c6bdae None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:10.585884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621235c-7fa1-4a76-9467-6409ec438c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:10.586497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621235c-7fa1-4a76-9467-6409ec438c38 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:10.586771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621235c-7fa1-4a76-9467-6409ec438c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:10.587046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621235c-7fa1-4a76-9467-6409ec438c38 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:10.587706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621235c-7fa1-4a76-9467-6409ec438c38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:10.589190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621235c-7fa1-4a76-9467-6409ec438c38 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:10.614349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 590/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:26:10 2022] POST /identity/v3/users => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:26:12.631788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af4b27-ec3b-4aa4-9c65-1b696a5d4bc3 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:12.632618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af4b27-ec3b-4aa4-9c65-1b696a5d4bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:12.632618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af4b27-ec3b-4aa4-9c65-1b696a5d4bc3 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:12.634690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 588/1178] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:12.642401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:12.642806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:12.642947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:12.662675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:12.663887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:12.735292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 591/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:26:12.758541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af4b27-ec3b-4aa4-9c65-1b696a5d4bc3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:12.873480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af4b27-ec3b-4aa4-9c65-1b696a5d4bc3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:12.911637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:12.912662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:12.913033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:12.913394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:12.928245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:26:12.933224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:26:12.934181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:26:12.935076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 589/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:26:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:26:12.940686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:12.951080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:13.043928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26648716-cf24-4c0e-ab82-2f0325051ed6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:13.087501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:13.088380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:13.088744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:13.089135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:13.090211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:13.090615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:13.092748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:13.101572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 592/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:26:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:13.110088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:13.126030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:13.223007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e70b24-b2ce-4445-bfb0-0fb83eb8fcf8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:13.266678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:13.267317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:13.267614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:13.267864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] PATH_INFO: `/v3/users/2bb86e30186e4ab9a50f7646fda073aa` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:13.281943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] RBAC: Authorizing `identity:get_user(user_id=2bb86e30186e4ab9a50f7646fda073aa)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:13.283391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:13.285126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 590/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:26:13 2022] GET /identity/v3/users/2bb86e30186e4ab9a50f7646fda073aa => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:13.291174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:13.362948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:13.405476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632586e6-1f18-495f-a7fe-055596295655 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:13.439150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:13.439752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:13.440022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:13.440272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:13.508846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:13.510299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:13.577521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:13.610269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 593/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:26:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:26:13.619534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:13.691467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:13.726273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac49b9-1e31-4de1-a43b-9125de55d194 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:13.761811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f374bd-6c0a-4438-83aa-2b736f60bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:13.762380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f374bd-6c0a-4438-83aa-2b736f60bc2a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:13.762656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f374bd-6c0a-4438-83aa-2b736f60bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:13.762923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f374bd-6c0a-4438-83aa-2b736f60bc2a None admin] PATH_INFO: `/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:13.764201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f374bd-6c0a-4438-83aa-2b736f60bc2a None admin] RBAC: Authorizing `identity:get_project(project_id=aedd0ee1ec0149b3a1cf589dda0273e5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:13.765659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f374bd-6c0a-4438-83aa-2b736f60bc2a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:13.767308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 591/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:26:13 2022] GET /identity/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:13.774312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:13.883660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428edb2-31ed-4b31-8192-c374e9480181 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:13.918689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:13.919286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:13.919553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:13.919813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:13.920589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=aedd0ee1ec0149b3a1cf589dda0273e5, user.id=2bb86e30186e4ab9a50f7646fda073aa) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:13.920907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:13.922356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5add9f-6b5d-43c5-a2d4-b61f6242cb63 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:13.927787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 594/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:26:13 2022] GET /identity/v3/role_assignments?user.id=2bb86e30186e4ab9a50f7646fda073aa&scope.project.id=aedd0ee1ec0149b3a1cf589dda0273e5&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:15.843785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087a002-31cf-416b-b5e3-39dfadce31a6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:15.844374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087a002-31cf-416b-b5e3-39dfadce31a6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:15.844374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087a002-31cf-416b-b5e3-39dfadce31a6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:15.845714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 592/1186] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:15.851958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:15.852692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:15.852919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:15.874229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:15.875422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:15.902873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:15.946076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 595/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:26:15.968024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087a002-31cf-416b-b5e3-39dfadce31a6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:16.077082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087a002-31cf-416b-b5e3-39dfadce31a6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:16.124202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:16.124934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:16.125212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:16.125484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:16.133072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:16.137234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:26:16.142507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:26:16.143625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:26:16.144736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 593/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:26:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:26:16.150390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:16.258923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273df1e-117d-4734-8db3-e79c39b6fbb3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:16.296412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:16.297117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:16.297374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:16.297630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:16.298335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:16.298631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:16.299987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:16.307936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 596/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:26:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:16.315578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:16.416302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8d8b4-6b91-4511-b4ad-4b8145296cc1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:16.450561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:16.451166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:16.451439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:16.451699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] PATH_INFO: `/v3/users/2bb86e30186e4ab9a50f7646fda073aa` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:16.452999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] RBAC: Authorizing `identity:get_user(user_id=2bb86e30186e4ab9a50f7646fda073aa)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:16.454415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:16.455974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 594/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:26:16 2022] GET /identity/v3/users/2bb86e30186e4ab9a50f7646fda073aa => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:16.464414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:16.492430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:16.567677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e521f-ff9e-417d-8ed8-96f0900aed0b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:16.601103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af68706-8348-4c1f-9ec4-306746c66cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:16.601726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af68706-8348-4c1f-9ec4-306746c66cee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:16.601986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af68706-8348-4c1f-9ec4-306746c66cee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:16.602248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af68706-8348-4c1f-9ec4-306746c66cee None admin] PATH_INFO: `/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:16.603402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af68706-8348-4c1f-9ec4-306746c66cee None admin] RBAC: Authorizing `identity:get_project(project_id=aedd0ee1ec0149b3a1cf589dda0273e5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:16.604851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af68706-8348-4c1f-9ec4-306746c66cee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:16.606466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 597/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:26:16 2022] GET /identity/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:26:16.615948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:16.693171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:16.732046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df90161-7609-4b79-92fd-26f916b3d952 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:16.766707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9e622-2e45-414f-92a2-4eb81b14025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:16.767361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9e622-2e45-414f-92a2-4eb81b14025c None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:16.767635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9e622-2e45-414f-92a2-4eb81b14025c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:16.767906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9e622-2e45-414f-92a2-4eb81b14025c None admin] PATH_INFO: `/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5/users/2bb86e30186e4ab9a50f7646fda073aa/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:16.769944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9e622-2e45-414f-92a2-4eb81b14025c None admin] RBAC: Authorizing `identity:create_grant(project_id=aedd0ee1ec0149b3a1cf589dda0273e5, user_id=2bb86e30186e4ab9a50f7646fda073aa, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:16.771363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9e622-2e45-414f-92a2-4eb81b14025c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:16.790790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 595/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:26:16 2022] PUT /identity/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5/users/2bb86e30186e4ab9a50f7646fda073aa/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:26:18.766343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1561-8b95-430d-8883-8188789720d9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:18.767234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1561-8b95-430d-8883-8188789720d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:18.767234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1561-8b95-430d-8883-8188789720d9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:18.769440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 598/1193] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:26:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:26:18.775811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:18.776540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:18.776702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:18.805792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:26:18.807577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:26:18.883587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 596/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:26:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:26:18.906587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1561-8b95-430d-8883-8188789720d9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:19.013031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:19.027769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1561-8b95-430d-8883-8188789720d9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:19.062553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:19.063194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:19.063456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:19.063720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:19.074716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:26:19.081149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:26:19.082314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:26:19.083240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 599/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:26:19.088733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:19.145449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:19.195942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78bc8d-f8b1-4af2-ac2d-69d9b08c53cb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:19.240024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:19.240645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:19.240950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:19.241206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:19.241920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:19.242217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:19.243580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:19.251803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 597/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:26:19.258190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:19.366732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6099f64-612c-4810-a342-e86174d998a8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:19.401198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:19.401826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:19.402099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:19.402366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] PATH_INFO: `/v3/users/2bb86e30186e4ab9a50f7646fda073aa` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:19.403557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] RBAC: Authorizing `identity:get_user(user_id=2bb86e30186e4ab9a50f7646fda073aa)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:19.405026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:19.406635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 600/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Sun Jul 31 18:26:19 2022] GET /identity/v3/users/2bb86e30186e4ab9a50f7646fda073aa => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:19.413328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:19.522689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d27ca-dde9-41f8-9ef1-e035bf834001 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:19.541802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:19.560062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:19.560667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:19.560971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:19.561239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:19.637177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:19.638545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:19.741642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 598/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:26:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:26:19.750982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:19.777884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:19.864127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0073-b60d-4067-aaff-854f6c29bb05 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:19.898126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a279768-a396-49d6-90a4-377a0477194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:19.898781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a279768-a396-49d6-90a4-377a0477194d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:19.899057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a279768-a396-49d6-90a4-377a0477194d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:19.899327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a279768-a396-49d6-90a4-377a0477194d None admin] PATH_INFO: `/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:19.900540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a279768-a396-49d6-90a4-377a0477194d None admin] RBAC: Authorizing `identity:get_project(project_id=aedd0ee1ec0149b3a1cf589dda0273e5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:19.902017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a279768-a396-49d6-90a4-377a0477194d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:19.903665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 601/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Sun Jul 31 18:26:19 2022] GET /identity/v3/projects/aedd0ee1ec0149b3a1cf589dda0273e5 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:19.910908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:26:19.930737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:20.017615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32292-8c6a-423f-bd71-bc28bfa039e1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:26:20.052572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:26:20.053193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:26:20.053473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:26:20.053735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:26:20.054512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=aedd0ee1ec0149b3a1cf589dda0273e5, user.id=2bb86e30186e4ab9a50f7646fda073aa) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:26:20.054805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:26:20.056209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79176-2568-42b7-b7fa-c3ed2211c9d6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:26:20.062184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 599/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:26:19 2022] GET /identity/v3/role_assignments?user.id=2bb86e30186e4ab9a50f7646fda073aa&scope.project.id=aedd0ee1ec0149b3a1cf589dda0273e5&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:26:29.893327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=55 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:26:30.059409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=59 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:14.746979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103c319-7d7d-4205-bbaa-31f18ab82eb1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:14.746979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103c319-7d7d-4205-bbaa-31f18ab82eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:14.746979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103c319-7d7d-4205-bbaa-31f18ab82eb1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:14.746979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 602/1201] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:14.752657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:14.753138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:14.753221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:14.773062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:14.774277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:14.856809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 600/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:14.878376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103c319-7d7d-4205-bbaa-31f18ab82eb1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:15.000571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103c319-7d7d-4205-bbaa-31f18ab82eb1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:15.046467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:15.047147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:15.047421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:15.047687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:15.060285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:15.062373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:15.079416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:15.080471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:15.086042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:15.086042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:15.086042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:15.086042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:15.086042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 603/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:15.086803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:15.213063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db19ac-0774-4d19-9173-3f9801ffa079 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:15.249627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:15.257918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:15.258876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:15.259344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:15.259803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:15.260693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:15.261027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:15.262649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccd489-6da5-48b5-bed3-aad6892f5c6c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:15.281919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 601/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:15.287908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:15.360290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:15.430110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54376119-9e66-4403-89c5-7a641a674258 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:15.473048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301f33c-d645-4c74-acb3-8f8226127a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:15.473783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301f33c-d645-4c74-acb3-8f8226127a9a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:15.474085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301f33c-d645-4c74-acb3-8f8226127a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:15.474370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301f33c-d645-4c74-acb3-8f8226127a9a None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:15.475214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301f33c-d645-4c74-acb3-8f8226127a9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:15.476889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301f33c-d645-4c74-acb3-8f8226127a9a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:15.510802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 604/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Sun Jul 31 18:27:15 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:27:17.639886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e5242-0b91-40ae-b239-69d07122c389 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:17.640448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e5242-0b91-40ae-b239-69d07122c389 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:17.640544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e5242-0b91-40ae-b239-69d07122c389 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:17.642744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 602/1206] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:17.648259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:17.648796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:17.648796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:17.669841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:17.671074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:17.747160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 605/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:17.769930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e5242-0b91-40ae-b239-69d07122c389 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:17.881570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e5242-0b91-40ae-b239-69d07122c389 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:17.921204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:17.921833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:17.922109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:17.922379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:17.933206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:27:17.938125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:17.939256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:17.940188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 603/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:27:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:17.945419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:17.985618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:18.054811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb478-ea86-46f7-b792-84563a64bbda None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:18.091379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:18.092032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:18.092309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:18.092575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:18.093345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:18.093641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:18.095278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:18.103568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 606/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:27:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:27:18.109640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:18.167424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:18.229051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132539-c9d2-49cc-bc6b-2c46cb5f8901 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:18.267154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:18.268570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:18.268874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:18.269150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:18.280037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:18.281867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:18.298066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:18.299136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:18.300363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:18.300363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:18.300363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:18.300363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:18.303957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 604/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:18.305064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:18.422056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:18.425474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aba1bf-8889-4105-add3-0052e6633852 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:18.473537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:18.474164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:18.474434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:18.474697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:18.475442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:18.475683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:18.477081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:18.492319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 607/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:18.498476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:18.620245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f2670-feb5-403d-a336-8168aaa0745a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:18.622073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:18.663807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:18.664616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:18.664931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:18.665229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:18.673828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:18.675350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:27:18.689990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:18.691058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:18.697214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 605/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:18.697841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:18.817340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689f730-7754-4467-a2c2-8a71afee1421 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:18.858351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:18.859514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:18.859886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:18.860257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:18.861314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:18.861871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:18.863640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:18.886694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 608/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Sun Jul 31 18:27:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:18.892811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:19.010943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db38c42-9fa3-4e52-a967-7118357388a5 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:19.050452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:19.051194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:19.051514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:19.051728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:19.088603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:19.136881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:19.138550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:19.252843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 606/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:27:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 18:27:19.259322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:19.281327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:19.368704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fa8ec-7ac1-4762-a2ec-f191e314c539 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:19.407508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:19.408234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:19.408488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:19.408788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:19.419323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:19.420954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:19.438552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:19.439597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:19.440843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:19.440843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:19.440843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:19.440843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:19.440843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 609/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:19.445918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:19.521779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:19.567780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95076-49ae-4130-a831-a4ae41697fe4 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:19.605397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:19.606012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:19.606284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:19.606545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:19.607291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:19.607589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:19.609054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:19.627401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 607/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:27:19.628723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:19.711451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:19.750043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531fa38-cf7e-4c7a-85fb-60cdbeedbd49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:19.788596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:19.789278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:19.789579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:19.789839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:19.799311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:19.800852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:27:19.815520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:19.816645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:19.817613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 610/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:19.823494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:19.970691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c1f41-840f-44e8-8b85-8996b8ff4cf7 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:20.003352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:20.018832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:20.019505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:20.019764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:20.020051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:20.020886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:20.021191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:20.023315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306d25d-16e4-4323-91fa-0784b7ad42f0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:20.040710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 608/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:20.047017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:20.176382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e5a16-879f-457d-8eaf-09e5083c16b0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:20.214720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:20.224635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:20.225177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:20.225451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:20.225724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:20.227063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=5686862728f34d64af248e12358541d8) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:20.227063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:20.228364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d434-456f-4240-8cd5-6dd5a0568d5d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:20.237741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 611/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:27:20 2022] GET /identity/v3/role_assignments?user.id=5686862728f34d64af248e12358541d8&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:22.220571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf06c8-3cd5-4398-b7e0-3396b302e32f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:22.220571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf06c8-3cd5-4398-b7e0-3396b302e32f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:22.221368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf06c8-3cd5-4398-b7e0-3396b302e32f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:22.224157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 609/1220] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:22.227745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:22.228176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:22.228274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:22.246731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:22.247916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:22.317149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 612/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:22.340081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf06c8-3cd5-4398-b7e0-3396b302e32f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:22.458671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf06c8-3cd5-4398-b7e0-3396b302e32f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:22.500886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:22.501368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:22.501639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:22.501989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:22.512469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:27:22.517544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:22.518952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:22.520020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 610/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:27:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:22.525421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:22.631629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57fe8f-c729-4e37-885f-c0b11ad936d6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:22.657984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:22.670519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:22.671189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:22.671543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:22.671779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:22.672485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:22.672806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:22.674201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:22.682930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 613/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:27:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:22.688989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:22.739994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:22.800392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d099-8abb-44b3-b87d-194915955d67 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:22.835713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:22.836379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:22.836648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:22.836949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:22.847591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:22.849063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:22.865930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:22.867253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:22.868521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:22.868521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:22.868521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:22.868521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:22.868521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 611/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:27:22.873385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:22.990701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6cc32-cc8c-4d27-8de4-55f34f86e5d6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:23.030682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:23.031306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:23.031577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:23.031844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:23.032518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:23.032842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:23.034204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:23.046571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 614/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:23.052525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:23.148967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:23.156420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b16f28-8bb6-4c2c-bfa4-4f131e563869 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:23.199470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:23.200109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:23.200387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:23.200646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:23.211392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:23.213129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:27:23.227225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:23.228699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:23.229667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 612/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:23.234589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:23.288420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:23.345027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712ebdd-503a-4f7c-b409-deaa806cc0b0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:23.381294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:23.381910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:23.382179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:23.382438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:23.383220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:23.383514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:23.384973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:23.402821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 615/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Sun Jul 31 18:27:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:23.409041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:23.524809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18774634-d53b-479e-97a6-1e2f11dd5b87 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:23.568920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:23.569593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:23.569985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:23.570262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:23.580594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:23.582063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:23.599364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:23.600447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:23.601445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:23.601445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:23.601445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:23.601445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:23.601445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 613/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:23.606760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:23.715863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c836e4-a1fc-41d9-936c-c3cb550ad13a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:23.718400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:23.753294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:23.753918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:23.754190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:23.754452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:23.755131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:23.755823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:23.757224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:23.768507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 616/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:23.774653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:23.785093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:23.881680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22c665-bd0e-40b6-a660-b8e703eb1871 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:23.920196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:23.921222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:23.921614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:23.922041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:23.930795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:23.932472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:27:23.946775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:23.948211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:23.949793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 614/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:27:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:27:23.954994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:24.059625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b381dda-0426-4121-b982-d31807641d91 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:24.095957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:24.096735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:24.097033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:24.097322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:24.098047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:24.098366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:24.099905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b7e10-654d-4af1-9c45-42ecf7f17532 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:24.113416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 617/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:27:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:24.119653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:24.193673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:24.241212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb86873-8af1-4a7e-8114-e0916488c29b None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:24.277496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cc96a-a980-4584-bc96-740c30cc45c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:24.278171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc96a-a980-4584-bc96-740c30cc45c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:24.278495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc96a-a980-4584-bc96-740c30cc45c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:24.278761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cc96a-a980-4584-bc96-740c30cc45c8 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/5686862728f34d64af248e12358541d8/roles/0ef50bc5478b4eaea3f2c254560df072` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:24.294562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cc96a-a980-4584-bc96-740c30cc45c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=5686862728f34d64af248e12358541d8, role_id=0ef50bc5478b4eaea3f2c254560df072)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:24.296053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cc96a-a980-4584-bc96-740c30cc45c8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:24.308441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 615/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:27:24 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/5686862728f34d64af248e12358541d8/roles/0ef50bc5478b4eaea3f2c254560df072 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:27:26.395640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643b7e1-d774-43f9-a62a-0523bab50a5b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:26.396219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643b7e1-d774-43f9-a62a-0523bab50a5b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:26.396219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643b7e1-d774-43f9-a62a-0523bab50a5b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:26.397757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 618/1233] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:26.403795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:26.404241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:26.404478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:26.425365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:26.426578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:26.504399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 616/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:26.532619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643b7e1-d774-43f9-a62a-0523bab50a5b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:26.558573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:26.640705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643b7e1-d774-43f9-a62a-0523bab50a5b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:26.675305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:26.675950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:26.676229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:26.676531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:26.687159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:27:26.692267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:26.693443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:26.694320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 619/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Sun Jul 31 18:27:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:26.700007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:26.750812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:26.805707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd51f04-b6a8-477f-898e-70db87389d40 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:26.842240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:26.842903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:26.843179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:26.843451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:26.844167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] RBAC: Adding query filter params (name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:26.844463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:26.845898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:26.855812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 617/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Sun Jul 31 18:27:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:27:26.861879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:26.960441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:26.981463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bf33f-cc7d-45f3-a973-ba040f2926ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:27.018447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:27.019109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:27.019457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:27.019731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:27.030602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:27.032220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:27.048562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:27.049718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:27.055555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:27.055555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:27.055555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:27.055555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:27.055555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 620/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:27.056709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:27.162385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76a46-c134-40d9-9ba3-0fb66fe1bf73 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:27.172524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:27.199711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:27.200325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:27.200598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:27.200882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:27.201549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:27.201864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:27.203226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:27.215551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 618/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:27.221681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:27.332309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fe74e-97ed-4d10-b46d-017f7ab6019d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:27.367400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:27.368349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:27.368616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:27.369080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:27.372423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:27.380053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:27.382627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:27:27.398100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:27.399603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:27.400821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 621/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:27.405352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:27.515114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf08ab6-7577-476e-a0c1-7c81b8e932bb None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:27.549660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:27.550275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:27.550551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:27.550819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:27.551591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:27.551891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:27.553348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:27.576826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 619/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Sun Jul 31 18:27:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:27.578189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:27.685011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640acee-3e07-448a-9512-7e9852f88d79 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:27.719965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:27.720583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:27.720881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:27.721147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:27.789691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:27.796968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:27.798608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:27.902800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 622/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:27:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:27:27.908516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:27.942358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:28.025511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed611b-eebb-49cc-949a-6d099d836af1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:28.062311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:28.062943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:28.063213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:28.063482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:28.074642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:28.076472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:28.092208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:28.093274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:28.094241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:28.094241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:28.094241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:28.094241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:28.094241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 620/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:28.100206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:28.200568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:28.208891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7984b7-407c-4a70-adce-1d0464500124 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:28.249942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:28.250559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:28.250829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:28.251090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:28.251764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:28.252057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:28.253605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:28.267085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 623/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:27:28.273694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:28.360988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:28.385589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f277-fa33-4023-b424-330f67152943 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:28.422415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:28.423090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:28.423319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:28.423584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:28.430633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:28.432073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:27:28.443832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:28.444982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:28.446322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 621/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:28.451868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:28.553190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1a719-31c4-454c-9584-365fe02e65d1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:28.593871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:28.594500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:28.594800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:28.595096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:28.595886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:28.596181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:28.597644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd74df-1469-4fa8-ac4c-00293f45c0e0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:28.611021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 624/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:27:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:28.617197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:28.728998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c626b6-962c-41f5-903c-a099e80a7a86 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:28.766351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:28.767065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:28.767428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:28.767881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:28.768809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] RBAC: Adding query filter params (role.id=0ef50bc5478b4eaea3f2c254560df072, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=5686862728f34d64af248e12358541d8) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:28.769207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:28.770928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c39b4-5200-40a4-a8a6-0c55aaeb18d2 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:28.777863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 622/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:27:28 2022] GET /identity/v3/role_assignments?user.id=5686862728f34d64af248e12358541d8&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=0ef50bc5478b4eaea3f2c254560df072 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:30.873931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db34d91-b023-4649-a565-a57bdeee7899 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:30.874594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db34d91-b023-4649-a565-a57bdeee7899 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:30.874594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db34d91-b023-4649-a565-a57bdeee7899 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:30.876050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 625/1247] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:30.883477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:30.884159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:30.884379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:30.891763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:30.907250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:30.908548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:30.986569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 623/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:31.011471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db34d91-b023-4649-a565-a57bdeee7899 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:31.032429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:31.128183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db34d91-b023-4649-a565-a57bdeee7899 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:31.165934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:31.167163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:31.167619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:31.168223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:31.181306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:27:31.188786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:31.189882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:31.190714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 626/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:27:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:27:31.196200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:31.304215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de8c03-8e1b-47b2-9787-2f80bcde8603 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:31.337575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:31.344321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:31.344993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:31.345283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:31.345532 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:31.346323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:31.346620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:31.347974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:31.357617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 624/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:27:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:31.363694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:31.462098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:31.488947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3b66-2141-43cf-b162-4b1b967475a2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:31.528499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:31.529602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:31.530222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:31.530563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:31.542466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:31.544447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:31.561182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:31.562124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:31.563005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:31.563005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:31.563005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:31.563005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:31.563005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 627/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:27:31.568968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:31.691205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd8e9-7753-436d-9387-5c91e7cde08a None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:31.730841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:31.731556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:31.731934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:31.732324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:31.733144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:31.733545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:31.735021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:31.748726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 625/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:31.755155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:31.867204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bcf6-16ca-4e5f-a7c0-244fc5e6065f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:31.896330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:31.905671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:31.906355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:31.906649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:31.906962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:31.916067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:31.917641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:27:31.930331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:31.931763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:31.932966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 628/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:31.938086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:32.012586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:32.072273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f0c3b-5d01-4c51-94cb-8c5abe13d666 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:32.114306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:32.114920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:32.115183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:32.115481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:32.116242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:32.116541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:32.118000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:32.138534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 626/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Sun Jul 31 18:27:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:32.144421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:32.268314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925c1d2-dbe4-43c2-a984-62d788c434e4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:32.305767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:32.306396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:32.306666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:32.306931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:32.372003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:32.387838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:32.389761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:32.496330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 629/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:27:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:27:32.502540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:32.638921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024ee7d-f352-49a3-ac2c-b178b29d8a5e None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:32.654588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:32.678743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:32.679376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:32.679649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:32.679915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:32.691141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:32.692630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:32.708098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:32.709278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:32.710269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:32.710269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:32.710269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:32.710269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:32.710269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 627/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:32.716394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:32.807712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:32.823023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053418af-1f0c-4e8d-b19a-8ec429ffb258 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:32.860509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:32.861177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:32.861484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:32.861795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:32.862431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:32.862733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:32.864139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:32.877915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 630/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:32.884166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:33.012736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e41ddb-32ee-47c3-afad-2a60a039f4b6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:33.055386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:33.056188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:33.056524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:33.056893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:33.065221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:33.066705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:27:33.079535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:33.080539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:33.081451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 628/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:27:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:33.089063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:33.210726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c6763-896c-4583-8927-eace32488ff3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:33.257016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:33.257649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:33.257922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:33.258189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:33.259038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:33.259377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:33.261206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc568fcd-c094-4673-9ef1-3c5afdb5ed66 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:33.275483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 631/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:33.282929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:33.311249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:33.396188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004509af-28b9-44ad-959b-4ccfa92d0970 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:33.435836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:33.436455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:33.436831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:33.437102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:33.437881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=5686862728f34d64af248e12358541d8) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:33.438202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:33.440018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302680b-7c2c-4333-b144-b21fae62af42 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:33.446780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 629/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:27:33 2022] GET /identity/v3/role_assignments?user.id=5686862728f34d64af248e12358541d8&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:27:35.439612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf991a-3fe9-4560-be82-480518d82fd1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:35.440253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf991a-3fe9-4560-be82-480518d82fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:35.440253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf991a-3fe9-4560-be82-480518d82fd1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:35.441536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 632/1261] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:35.447310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:35.447870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:35.447870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:35.468547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:35.469857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:35.544168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 630/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:35.566494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf991a-3fe9-4560-be82-480518d82fd1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:35.578222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:35.677226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf991a-3fe9-4560-be82-480518d82fd1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:35.712309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:35.712989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:35.713256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:35.713522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:35.723857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:27:35.728737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:35.729900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:35.730908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 633/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:27:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:35.736970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:35.759178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:35.839997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccad41-0570-4764-a5c4-a99da899c6f5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:35.873550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:35.874188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:35.874459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:35.874720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:35.875456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:35.875743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:35.877129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:35.885361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 631/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:27:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:35.891991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:35.973849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:36.007944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0b8e-66ff-4cc6-8afe-f0705603b927 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:36.055575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:36.056202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:36.056470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:36.056734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:36.067072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:36.068519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:36.086625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:36.087667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:36.088846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:36.088846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:36.088846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:36.088846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:36.088846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 634/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:27:36.094782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:36.176543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:36.199190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7710-39d3-4e79-a74e-6ed7163d50f8 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:36.237614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:36.238518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:36.238845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:36.239211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:36.240223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:36.240627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:36.242712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:36.257715 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 632/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:36.263682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:36.369158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f65b6-9a4a-458d-a23e-3b8e79d5be5a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:36.391740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:36.410787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:36.411420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:36.411692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:36.411951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:36.419910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:36.421413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:27:36.432847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:36.433994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:36.434934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 635/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:36.440329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:36.542853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f356b8-fb78-4ef6-bf5f-56f4af8b8714 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:36.589215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:36.589820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:36.590096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:36.590365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:36.591145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:36.591452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:36.592891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:36.601676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:36.612832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 633/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Sun Jul 31 18:27:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:36.618828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:36.722247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ef440-0b44-432e-9f20-bbf46e42f191 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:36.758343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:36.758959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:36.759227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:36.759490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:36.770236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:36.771663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:36.786231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:36.787248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:36.788728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:36.788728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:36.788728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:36.788728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:36.788728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 636/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 18:27:36.795493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:36.901735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c444c8-83f2-4269-a06b-3d5835a4a2ef None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:36.939226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:36.939942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:36.940252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:36.940542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:36.941361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:36.941694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:36.943212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:36.955799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 634/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:36.961699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:36.994514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:37.067055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83257497-a279-40f9-bab0-92f3fbf7ea7e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:37.101332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:37.101944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:37.102212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:37.102472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:37.109042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:37.110432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:27:37.121679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:37.122656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:37.123573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 637/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:37.130031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:37.199378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:37.243857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eb1ff-16aa-4217-b7df-adc5e755cfdc None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:37.278724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:37.279247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:37.279517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:37.279786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:37.280586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:37.280915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:37.282325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4440e1-2dc5-4735-a708-6a76462e291c None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:37.294267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 635/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:37.304151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:37.402459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:37.412675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c272488-beef-473d-abc5-afcdd324f2ad None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:37.451502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85d29e-cd8c-4729-9971-86b9b6e7c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:37.452265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85d29e-cd8c-4729-9971-86b9b6e7c110 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:37.452538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85d29e-cd8c-4729-9971-86b9b6e7c110 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:37.452793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85d29e-cd8c-4729-9971-86b9b6e7c110 None admin] PATH_INFO: `/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/5686862728f34d64af248e12358541d8/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:37.454897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85d29e-cd8c-4729-9971-86b9b6e7c110 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf16437efd0f4e3188059acaacd8a01a, user_id=5686862728f34d64af248e12358541d8, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:37.456368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85d29e-cd8c-4729-9971-86b9b6e7c110 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:37.471405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 638/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Sun Jul 31 18:27:37 2022] PUT /identity/v3/projects/cf16437efd0f4e3188059acaacd8a01a/users/5686862728f34d64af248e12358541d8/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:27:39.439274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273936ac-1ed5-49e9-9ab0-a1be0924444c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:39.439814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273936ac-1ed5-49e9-9ab0-a1be0924444c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:39.439814 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273936ac-1ed5-49e9-9ab0-a1be0924444c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:39.442044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 636/1274] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:27:39.450765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:39.451179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:39.451373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:39.476851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:39.478023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:39.546836 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 639/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:39.575421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273936ac-1ed5-49e9-9ab0-a1be0924444c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:39.689563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273936ac-1ed5-49e9-9ab0-a1be0924444c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:39.706836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:39.726134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:39.726787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:39.727119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:39.727332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:39.737388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 18:27:39.742365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:39.743306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:39.744165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 637/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Sun Jul 31 18:27:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:39.749778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:39.838150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:39.862659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3fff-de86-4e70-a8da-1242d8d12f93 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:39.897857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:39.898475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:39.898754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:39.899028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:39.899740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:39.900049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:39.901456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:39.909962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 640/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Sun Jul 31 18:27:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:39.916900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:40.030008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d77c88-b793-4db1-b3de-4e47a002c8c1 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:40.065117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:40.065748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:40.066025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:40.066293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:40.076599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:40.078115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:40.093810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:40.094793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:40.096020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:40.096020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:40.096020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:40.096020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:40.096020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 638/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:40.103730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:40.207871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16face-81bc-45d3-8d33-a6e4779381cf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:40.246846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:40.247913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:40.248348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:40.248912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:40.249760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:40.250112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:40.251558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:40.264875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 641/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:40.271276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:40.315581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:40.387488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0184402-75c2-4f9a-a65e-da6a6f251ef6 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:40.422462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:40.423084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:40.423359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:40.423628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:40.431220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:40.432648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 18:27:40.444087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:40.446271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:40.447235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 639/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:40.451355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:40.455631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:40.550501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe06d6d-4e6d-4c93-bed2-dd997efdc570 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:40.589621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:40.590468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:40.590886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:40.591302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:40.592593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:40.593035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:40.595287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:40.617589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 642/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Sun Jul 31 18:27:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:40.623528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:40.718067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:40.733920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fc28-f093-4a80-b99b-06b4568f5626 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:40.772162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:40.772756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:40.773065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:40.773331 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:40.842710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:40.844086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:40.943407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:40.958907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 640/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Sun Jul 31 18:27:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 18:27:40.965469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:41.030298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:41.072742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da470719-d5a2-4d81-ac55-820f3a21321e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:41.108357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:41.108937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:41.109178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:41.109476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:41.119819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:41.121287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 18:27:41.136459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:41.137527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:41.143798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 18:27:41.143798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 18:27:41.143798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 18:27:41.143798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:41.143798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 643/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:41.144191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:41.250867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd025ffb-09be-44b4-9aa0-97c46e552615 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:41.295323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:41.296130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:41.296409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:41.296675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:41.297396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:41.297691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:41.299055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:41.310760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 641/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Sun Jul 31 18:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:41.316785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:41.421679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a2d1-c789-444a-be5b-0db31e435f5d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:41.435192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:41.457158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:41.457802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:41.458058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:41.458329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:41.465285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:41.467272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 18:27:41.478998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:41.480023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 18:27:41.481210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jul 31 18:27:41.484960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 644/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Sun Jul 31 18:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:41.486719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:41.543198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:41.596422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565fede-175e-4ee3-943d-379c617576ce None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:41.634839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:41.635447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:41.635720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:41.635988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:41.636733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:41.637071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:41.638500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582e437-4dc6-4369-8c7b-6dfa6b5c5fa7 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:41.650660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 642/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Sun Jul 31 18:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:41.657678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:41.782857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966da91-9761-4c4f-9642-811da9c9df73 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:41.818201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:41.818803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:41.819074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:41.819344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:41.820123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] RBAC: Adding query filter params (role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1, scope.project.id=cf16437efd0f4e3188059acaacd8a01a, user.id=5686862728f34d64af248e12358541d8) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:41.820428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:41.821901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e2ece-7024-42d8-bdb0-790ce8ae7148 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:41.827859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 645/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Sun Jul 31 18:27:41 2022] GET /identity/v3/role_assignments?user.id=5686862728f34d64af248e12358541d8&scope.project.id=cf16437efd0f4e3188059acaacd8a01a&role.id=e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:43.801577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ab683-481e-4e48-a096-b3e5bbf527ad None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:43.802427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ab683-481e-4e48-a096-b3e5bbf527ad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:43.802427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ab683-481e-4e48-a096-b3e5bbf527ad None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:43.804559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 643/1288] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:43.813341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:43.813759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:43.813894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:43.843083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:43.844267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:43.878608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:43.918965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 646/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:43.943775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ab683-481e-4e48-a096-b3e5bbf527ad None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:44.050495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ab683-481e-4e48-a096-b3e5bbf527ad None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:44.060659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:44.085166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:44.085806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:44.086083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:44.086353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:44.086977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:44.088339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:44.097496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:44.098663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:44.099753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:44.100921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 644/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:27:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:44.105095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:44.223232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cb549-6006-42e2-ab99-b521e81f7734 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:44.264203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:44.264905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:44.265182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:44.265448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:44.266118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:44.266416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:44.267799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00b79f-23a5-423f-a978-57bdcbbd03bf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:44.272956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 647/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Sun Jul 31 18:27:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:44.282444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:44.396322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679008a-3b4a-4cf5-ad84-58348e5c5915 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:44.437456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:44.438064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:44.438353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:44.438621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] PATH_INFO: `/v3/services` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:44.439297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:44.439598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:44.441000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01e005-0f4c-491c-b3f8-daa04e0e334d None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:44.445675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 645/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Sun Jul 31 18:27:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:46.406244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3421cdc-bb1d-4934-9339-4c22b78a28b2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:46.407182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3421cdc-bb1d-4934-9339-4c22b78a28b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:46.407182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3421cdc-bb1d-4934-9339-4c22b78a28b2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:46.409605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 648/1293] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:46.415369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb019f9e-a3a9-457f-9a2b-d94ffc3726e4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:46.415831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb019f9e-a3a9-457f-9a2b-d94ffc3726e4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:46.415938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb019f9e-a3a9-457f-9a2b-d94ffc3726e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:46.437903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bb019f9e-a3a9-457f-9a2b-d94ffc3726e4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:46.439117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bb019f9e-a3a9-457f-9a2b-d94ffc3726e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:46.470897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:46.510791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 646/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:46.533302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3421cdc-bb1d-4934-9339-4c22b78a28b2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:46.574819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:46.648607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3421cdc-bb1d-4934-9339-4c22b78a28b2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:46.682961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8e826-0301-4763-90e8-6fca21030c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:46.683566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8e826-0301-4763-90e8-6fca21030c49 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:46.683837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8e826-0301-4763-90e8-6fca21030c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:46.684102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8e826-0301-4763-90e8-6fca21030c49 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:46.684792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8e826-0301-4763-90e8-6fca21030c49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:46.686239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8e826-0301-4763-90e8-6fca21030c49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:46.704526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 649/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Sun Jul 31 18:27:46 2022] POST /identity/v3/services => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:27:48.671154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87777-b2bf-447d-8c2f-056a520907a2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:48.672033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87777-b2bf-447d-8c2f-056a520907a2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:48.672033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db87777-b2bf-447d-8c2f-056a520907a2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:48.674187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 647/1296] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:48.682772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:48.683194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:48.683334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:48.709313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:48.710509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:48.785664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 650/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:48.808573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87777-b2bf-447d-8c2f-056a520907a2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:48.929262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db87777-b2bf-447d-8c2f-056a520907a2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:48.966863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:48.967605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:48.967922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:48.968199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:48.968908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:48.970493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:48.980843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:48.982071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:48.983034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 648/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:27:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:48.988643 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:49.008555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:49.090030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a912d-1855-460a-b44d-b6911f592e2a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:49.125555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:49.126169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:49.126446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:49.126715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:49.127397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:49.127697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:49.129110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:49.135103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 651/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Sun Jul 31 18:27:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:49.142121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:49.180876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:49.253954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283301d2-7cea-4b8e-ba76-27e874ac25e0 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:49.288422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:49.289063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:49.289337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:49.289602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:49.290360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] RBAC: Adding query filter params (interface=public, service_id=606dfd2c0b0c4cfbb031932042554af9, region_id=RegionOne) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:49.290660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:49.292029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3632-7771-4d80-a3f8-de442594bad0 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:49.297192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 649/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Sun Jul 31 18:27:49 2022] GET /identity/v3/endpoints?service_id=606dfd2c0b0c4cfbb031932042554af9&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:49.303090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:49.395059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:49.416473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be68fd6-4c59-4605-aca9-3aecf5f094c5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:49.451085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40d86d-39b6-4e51-bc9b-b0059b9a3d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:49.451642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40d86d-39b6-4e51-bc9b-b0059b9a3d44 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:49.451945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40d86d-39b6-4e51-bc9b-b0059b9a3d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:49.452215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40d86d-39b6-4e51-bc9b-b0059b9a3d44 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:49.452888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40d86d-39b6-4e51-bc9b-b0059b9a3d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:49.454271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40d86d-39b6-4e51-bc9b-b0059b9a3d44 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:49.461159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 652/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Sun Jul 31 18:27:49 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:27:51.417281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010c653-6643-4ab8-b6ea-1681e8c33e9a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:51.418175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010c653-6643-4ab8-b6ea-1681e8c33e9a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:51.418175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010c653-6643-4ab8-b6ea-1681e8c33e9a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:51.420336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 650/1302] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:27:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:27:51.426567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:51.427301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:51.427481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:51.451010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:27:51.452195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:27:51.525907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 653/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:27:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 18:27:51.548263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010c653-6643-4ab8-b6ea-1681e8c33e9a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:51.650087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:51.658420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010c653-6643-4ab8-b6ea-1681e8c33e9a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:51.697628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:51.698258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:51.698530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:51.698804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:51.699409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:51.700788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 18:27:51.710471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 18:27:51.711495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application with Lock( Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self._enter() Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application return self.creator() Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application created_value = creator( Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: ERROR keystone.server.flask.application Jul 31 18:27:51.712353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 651/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Sun Jul 31 18:27:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 18:27:51.720126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:51.801638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:27:51.838096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bea534-c7a9-43e5-8e8a-d67835d69387 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:51.874695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:51.875304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:51.875579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:51.875843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:51.876512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:27:51.876805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:51.878198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba0d43-61a3-47db-a0a1-54a0a29a1be8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:51.883838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 654/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Sun Jul 31 18:27:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:27:51.891784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:27:52.007847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8e8b5-8b46-43e4-ac01-bce65cd44743 None admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:27:52.043394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523feec-9f38-469c-86bd-aaa51f166735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:27:52.044009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523feec-9f38-469c-86bd-aaa51f166735 None admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:27:52.044285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523feec-9f38-469c-86bd-aaa51f166735 None admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:27:52.044551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523feec-9f38-469c-86bd-aaa51f166735 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:27:52.045309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523feec-9f38-469c-86bd-aaa51f166735 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:27:52.046677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523feec-9f38-469c-86bd-aaa51f166735 None admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:27:52.068652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 652/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:27:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:28:01.880793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=30 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:28:02.064099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:28:54.510430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829b058-0369-4eaf-a2be-4d28e5dac959 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:54.511471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829b058-0369-4eaf-a2be-4d28e5dac959 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:54.511947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829b058-0369-4eaf-a2be-4d28e5dac959 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:54.517503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 655/1307] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:28:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:28:54.518778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1cc49-0e4f-497c-8d2b-29493d9a3e0b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:54.519181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1cc49-0e4f-497c-8d2b-29493d9a3e0b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:54.519355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1cc49-0e4f-497c-8d2b-29493d9a3e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:54.603051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6cf1cc49-0e4f-497c-8d2b-29493d9a3e0b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:28:54.604247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6cf1cc49-0e4f-497c-8d2b-29493d9a3e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:54.736807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 653/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:28:54.793256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab852-0d66-469d-9888-38e416d2dc3f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:54.793709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab852-0d66-469d-9888-38e416d2dc3f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:54.793977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab852-0d66-469d-9888-38e416d2dc3f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:54.797923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 656/1309] 10.222.0.49 () {54 vars in 965 bytes} [Sun Jul 31 18:28:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:28:54.808674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788023d-a5aa-416f-a703-55a111d6b84b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:54.809168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788023d-a5aa-416f-a703-55a111d6b84b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:54.809781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788023d-a5aa-416f-a703-55a111d6b84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:54.903278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3788023d-a5aa-416f-a703-55a111d6b84b None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:28:54.904646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3788023d-a5aa-416f-a703-55a111d6b84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:54.932387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:28:54.997037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 654/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Sun Jul 31 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:28:55.006609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab852-0d66-469d-9888-38e416d2dc3f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:28:55.008372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d7ab852-0d66-469d-9888-38e416d2dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:55.128468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab852-0d66-469d-9888-38e416d2dc3f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:28:55.210069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:28:55.210783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:55.211115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:55.211445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:55.213187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:55.276283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:28:55.332046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:28:55.333626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8225e-426e-40c9-bea6-8199fe8b169b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:28:55.353021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 657/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:28:57.633360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774c6b8-e7c4-438f-a553-013d094f7c54 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:57.633933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774c6b8-e7c4-438f-a553-013d094f7c54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:57.633933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774c6b8-e7c4-438f-a553-013d094f7c54 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:57.635431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 655/1312] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:28:57.640631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad076-c171-46d5-9402-aefa1b32ff96 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:57.641113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad076-c171-46d5-9402-aefa1b32ff96 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:57.641193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad076-c171-46d5-9402-aefa1b32ff96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:57.717788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d02ad076-c171-46d5-9402-aefa1b32ff96 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:28:57.719058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d02ad076-c171-46d5-9402-aefa1b32ff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:57.825941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 658/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:28:57.877336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3c4b-b875-4159-90f6-0384de86a0fc None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:57.877571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3c4b-b875-4159-90f6-0384de86a0fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:57.877884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff3c4b-b875-4159-90f6-0384de86a0fc None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:57.879399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 656/1314] 10.222.0.49 () {54 vars in 965 bytes} [Sun Jul 31 18:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:28:57.885773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb2c2-61d7-43fe-b879-d43c937b5372 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:57.885986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb2c2-61d7-43fe-b879-d43c937b5372 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:57.886254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb2c2-61d7-43fe-b879-d43c937b5372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:57.899829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:28:57.979668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-94dfb2c2-61d7-43fe-b879-d43c937b5372 None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:28:57.981269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-94dfb2c2-61d7-43fe-b879-d43c937b5372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:58.068707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 659/1315] 10.222.0.49 () {58 vars in 1062 bytes} [Sun Jul 31 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:28:58.077930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3c4b-b875-4159-90f6-0384de86a0fc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:28:58.079472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-77ff3c4b-b875-4159-90f6-0384de86a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:58.187845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff3c4b-b875-4159-90f6-0384de86a0fc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:28:58.266247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:28:58.269094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:28:58.269727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:28:58.270002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:28:58.270256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:28:58.272482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:28:58.393443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:28:58.394895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d64fb-8ad4-48be-bdbd-f9ce546c45ee service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:28:58.413053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 657/1316] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:29:00.928384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9412f-dc19-47bf-83a4-338a00f2657c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:00.929009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9412f-dc19-47bf-83a4-338a00f2657c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:00.929009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9412f-dc19-47bf-83a4-338a00f2657c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:00.930304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 660/1317] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:00.936717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a786b0e-a280-42a1-96f8-cf45bece6c54 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:00.937548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a786b0e-a280-42a1-96f8-cf45bece6c54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:00.937640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a786b0e-a280-42a1-96f8-cf45bece6c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:01.011957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2a786b0e-a280-42a1-96f8-cf45bece6c54 None None] MFA Rules not processed for user `1e97c57f35af41ef8cc68743b7a473c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:01.013194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2a786b0e-a280-42a1-96f8-cf45bece6c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:01.193110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:01.239666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 658/1318] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:01.283726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9412f-dc19-47bf-83a4-338a00f2657c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:01.286332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2ae9412f-dc19-47bf-83a4-338a00f2657c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:01.398137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9412f-dc19-47bf-83a4-338a00f2657c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:01.400987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:01.489383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:01.489919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:01.490196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:01.490458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:01.492084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:01.606524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:01.607969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cebec6-f21e-43b7-98d4-921c47e04910 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:01.626155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 661/1319] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:04.238214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1fc0-d010-4407-b0ff-b051ecf924a0 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:04.238724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1fc0-d010-4407-b0ff-b051ecf924a0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:04.238724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1fc0-d010-4407-b0ff-b051ecf924a0 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:04.240115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 659/1320] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:04.245310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a05020-1daa-469d-96db-f7c5df3b02fb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:04.245736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a05020-1daa-469d-96db-f7c5df3b02fb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:04.245826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a05020-1daa-469d-96db-f7c5df3b02fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:04.286822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:04.314144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-69a05020-1daa-469d-96db-f7c5df3b02fb None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:04.315354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69a05020-1daa-469d-96db-f7c5df3b02fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:04.421908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 662/1321] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:04.463747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1fc0-d010-4407-b0ff-b051ecf924a0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:04.466231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-768e1fc0-d010-4407-b0ff-b051ecf924a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:04.592309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1fc0-d010-4407-b0ff-b051ecf924a0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:04.677536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:04.678402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:04.678786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:04.679114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:04.681720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:04.690630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:04.796485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:04.797971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2173d-4c4b-43b9-bbd0-dc07f04d0282 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:04.816791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 660/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:07.474110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d4d8-5745-4906-8a65-b185cb361abb None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:07.475092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d4d8-5745-4906-8a65-b185cb361abb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:07.475380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d4d8-5745-4906-8a65-b185cb361abb None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:07.476852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 663/1323] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:07.482476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec672db0-f1f7-4b68-ac1a-d1922806abfc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:07.482941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec672db0-f1f7-4b68-ac1a-d1922806abfc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:07.483048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec672db0-f1f7-4b68-ac1a-d1922806abfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:07.550273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ec672db0-f1f7-4b68-ac1a-d1922806abfc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:07.551496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ec672db0-f1f7-4b68-ac1a-d1922806abfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:07.609340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:07.654829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 661/1324] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:07.691506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70d4d8-5745-4906-8a65-b185cb361abb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:07.693125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc70d4d8-5745-4906-8a65-b185cb361abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:07.805239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70d4d8-5745-4906-8a65-b185cb361abb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:07.819262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:07.882711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:07.883336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:07.883606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:07.883866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:07.885689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:08.001150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:08.002606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe69da8-ace5-4886-82f9-775d2db0947a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:08.021673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 664/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:10.101628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde107-a0a6-46f9-86e2-ecf837965f47 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:10.102249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde107-a0a6-46f9-86e2-ecf837965f47 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:10.102249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde107-a0a6-46f9-86e2-ecf837965f47 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:10.103532 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 662/1326] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:10.108804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79993127-31e5-4a39-aba2-9f6bfdef1c0c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:10.109229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79993127-31e5-4a39-aba2-9f6bfdef1c0c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:10.109404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79993127-31e5-4a39-aba2-9f6bfdef1c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:10.161088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:10.179896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-79993127-31e5-4a39-aba2-9f6bfdef1c0c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:10.181155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-79993127-31e5-4a39-aba2-9f6bfdef1c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:10.278036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 665/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:10.313763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde107-a0a6-46f9-86e2-ecf837965f47 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:10.315287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-87cde107-a0a6-46f9-86e2-ecf837965f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:10.421679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde107-a0a6-46f9-86e2-ecf837965f47 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:10.501744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:10.502306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:10.502540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:10.502832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:10.504537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:10.523478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:10.622736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:10.624161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b05c89-1595-4f58-91ee-155962b8cd2d service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:10.643450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 663/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:12.667501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e458c-0f4f-49a0-b265-08e8791b4d5d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:12.668085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e458c-0f4f-49a0-b265-08e8791b4d5d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:12.668085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e458c-0f4f-49a0-b265-08e8791b4d5d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:12.669439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 666/1329] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:12.674949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd7a4d-852b-45fe-9728-af06c5b983e6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:12.675430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd7a4d-852b-45fe-9728-af06c5b983e6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:12.675524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd7a4d-852b-45fe-9728-af06c5b983e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:12.741885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aacd7a4d-852b-45fe-9728-af06c5b983e6 None None] MFA Rules not processed for user `1e97c57f35af41ef8cc68743b7a473c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:12.743079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aacd7a4d-852b-45fe-9728-af06c5b983e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:12.810164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:12.845025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 664/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:12.882439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e458c-0f4f-49a0-b265-08e8791b4d5d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:12.884024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-139e458c-0f4f-49a0-b265-08e8791b4d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:13.009476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e458c-0f4f-49a0-b265-08e8791b4d5d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:13.030231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:13.088739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:13.089318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:13.089552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:13.089850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:13.091462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:13.207815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:13.209330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c118f-3928-4356-bae8-676dbd351e8e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:13.228268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 667/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:15.603815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38842e2b-6c68-4320-a858-af9973a9e160 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:15.604377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38842e2b-6c68-4320-a858-af9973a9e160 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:15.604377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38842e2b-6c68-4320-a858-af9973a9e160 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:15.605762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 665/1332] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:15.612112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bfa4c-de60-4190-b9ad-f3aa9b68d809 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:15.612526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bfa4c-de60-4190-b9ad-f3aa9b68d809 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:15.612669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bfa4c-de60-4190-b9ad-f3aa9b68d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:15.690534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:15.695356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-016bfa4c-de60-4190-b9ad-f3aa9b68d809 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:15.697564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-016bfa4c-de60-4190-b9ad-f3aa9b68d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:15.797876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 668/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:15.845611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38842e2b-6c68-4320-a858-af9973a9e160 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:15.847561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-38842e2b-6c68-4320-a858-af9973a9e160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:15.964110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38842e2b-6c68-4320-a858-af9973a9e160 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:16.045053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:16.045673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:16.045947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:16.046240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:16.047846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:16.076939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:16.177466 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:16.178910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5097f5-b230-4c27-9e26-87b18454c34d service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:16.199632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 666/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:18.250450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc1942-dfc9-4fe9-831e-4f9fed3f4bad None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:18.250953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc1942-dfc9-4fe9-831e-4f9fed3f4bad None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:18.250953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc1942-dfc9-4fe9-831e-4f9fed3f4bad None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:18.252370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 669/1335] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:18.258033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5a4b-e6bc-4cc3-aa81-fde616568426 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:18.258499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5a4b-e6bc-4cc3-aa81-fde616568426 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:18.258600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5a4b-e6bc-4cc3-aa81-fde616568426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:18.327260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-587e5a4b-e6bc-4cc3-aa81-fde616568426 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:18.328477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-587e5a4b-e6bc-4cc3-aa81-fde616568426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:18.406058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:18.439277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 667/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:18.482979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc1942-dfc9-4fe9-831e-4f9fed3f4bad None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:18.484868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2dbc1942-dfc9-4fe9-831e-4f9fed3f4bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:18.594023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc1942-dfc9-4fe9-831e-4f9fed3f4bad None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:18.639232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:18.697623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:18.698255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:18.698583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:18.698852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:18.700475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:18.819646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:18.821041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac3ecd-aa61-4dcf-b4ef-f272bb483e14 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:18.846560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 670/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:21.312120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb017a-6418-4821-9101-13c5be0b054a None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:21.312700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb017a-6418-4821-9101-13c5be0b054a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:21.312700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb017a-6418-4821-9101-13c5be0b054a None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:21.314131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 668/1338] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:21.320437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183df14-503f-4cf9-9840-f700cafb4cfc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:21.321199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183df14-503f-4cf9-9840-f700cafb4cfc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:21.321364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183df14-503f-4cf9-9840-f700cafb4cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:21.396019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1183df14-503f-4cf9-9840-f700cafb4cfc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:21.397251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1183df14-503f-4cf9-9840-f700cafb4cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:21.405691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:21.502374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 671/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:21.540011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb017a-6418-4821-9101-13c5be0b054a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:21.541891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7dbb017a-6418-4821-9101-13c5be0b054a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:21.646602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb017a-6418-4821-9101-13c5be0b054a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:21.730384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:21.730993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:21.731257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:21.731518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:21.733160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:21.772205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:21.858100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:21.859542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8b42-a761-48ea-8ea7-46f96c04a609 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:21.878252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 669/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:25.878820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56233883-5d37-4f66-83d7-8d6c570f1c6a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:25.879351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56233883-5d37-4f66-83d7-8d6c570f1c6a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:25.879351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56233883-5d37-4f66-83d7-8d6c570f1c6a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:25.881177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 672/1341] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:25.886513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c1598-bc65-448a-a654-a0c000c27cb9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:25.886993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c1598-bc65-448a-a654-a0c000c27cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:25.886993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c1598-bc65-448a-a654-a0c000c27cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:25.954026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-931c1598-bc65-448a-a654-a0c000c27cb9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:25.955213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-931c1598-bc65-448a-a654-a0c000c27cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:26.050655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:26.071964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 670/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:26.110406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56233883-5d37-4f66-83d7-8d6c570f1c6a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:26.111921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-56233883-5d37-4f66-83d7-8d6c570f1c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:26.221241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56233883-5d37-4f66-83d7-8d6c570f1c6a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:26.261863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:26.299882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:26.300520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:26.300889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:26.301216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:26.302898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:26.415818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:26.417404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548619c-2e10-4b62-b5b1-eeaf46de78e2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:26.437449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 673/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:29.033964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d992e4-2df1-4196-8a6b-524781a31588 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:29.034590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d992e4-2df1-4196-8a6b-524781a31588 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:29.034590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d992e4-2df1-4196-8a6b-524781a31588 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:29.035907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 671/1344] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:29.041693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94638b-a878-46f7-b895-36a62bb51a1e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:29.042245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94638b-a878-46f7-b895-36a62bb51a1e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:29.042245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94638b-a878-46f7-b895-36a62bb51a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:29.117533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0e94638b-a878-46f7-b895-36a62bb51a1e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:29.118750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0e94638b-a878-46f7-b895-36a62bb51a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:29.139967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:29.226381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 674/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:29.262985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d992e4-2df1-4196-8a6b-524781a31588 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:29.264591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-52d992e4-2df1-4196-8a6b-524781a31588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:29.370645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d992e4-2df1-4196-8a6b-524781a31588 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:29.453660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:29.454274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:29.454547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:29.454809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:29.456405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:29.501614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:29.573414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:29.575499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b064b9-c420-4cce-90ed-fa61100af6ab service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:29.595170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 672/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:33.792174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3307-eeb9-4421-9790-88fb07a8be37 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:33.792701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3307-eeb9-4421-9790-88fb07a8be37 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:33.792701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a3307-eeb9-4421-9790-88fb07a8be37 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:33.794112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 675/1347] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:33.799461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740b61-c0d1-4d77-bb05-605c4efd46e1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:33.799899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740b61-c0d1-4d77-bb05-605c4efd46e1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:33.799999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740b61-c0d1-4d77-bb05-605c4efd46e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:33.886768 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-64740b61-c0d1-4d77-bb05-605c4efd46e1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:33.887992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-64740b61-c0d1-4d77-bb05-605c4efd46e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:33.992645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:33.995941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 673/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:34.034427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3307-eeb9-4421-9790-88fb07a8be37 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:34.035965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f6a3307-eeb9-4421-9790-88fb07a8be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:34.147676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a3307-eeb9-4421-9790-88fb07a8be37 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:34.196334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:34.223522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:34.224232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:34.224574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:34.224945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:34.226647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:34.349532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:34.351504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b783a-5691-47af-bd5b-c81c30c122a4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:34.370980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 676/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:39.200066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605f119-bdcc-479e-9424-1cfbb7682485 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:39.200541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605f119-bdcc-479e-9424-1cfbb7682485 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:39.200655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605f119-bdcc-479e-9424-1cfbb7682485 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:39.202608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 674/1350] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:39.207730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97e4d6-1ba3-4c48-a7b8-77696d98ab1f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:39.208151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97e4d6-1ba3-4c48-a7b8-77696d98ab1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:39.208289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97e4d6-1ba3-4c48-a7b8-77696d98ab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:39.332422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ab97e4d6-1ba3-4c48-a7b8-77696d98ab1f None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:39.333534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab97e4d6-1ba3-4c48-a7b8-77696d98ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:39.358981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:39.482251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 677/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:43.550217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858afa7-c3d2-4023-a6a1-5204040fa7ce None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:43.551042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858afa7-c3d2-4023-a6a1-5204040fa7ce None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:43.551491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858afa7-c3d2-4023-a6a1-5204040fa7ce None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:43.553097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 675/1352] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:43.558066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212569d-ddb2-46ce-ae42-bc1898bcfead None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:43.558522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212569d-ddb2-46ce-ae42-bc1898bcfead None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:43.558860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212569d-ddb2-46ce-ae42-bc1898bcfead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:43.649724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0212569d-ddb2-46ce-ae42-bc1898bcfead None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:43.650949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0212569d-ddb2-46ce-ae42-bc1898bcfead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:43.723725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:43.769865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 678/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:45.001889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c3725-ce55-487c-a298-9ce8879e5a77 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:45.002452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c3725-ce55-487c-a298-9ce8879e5a77 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:45.002452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c3725-ce55-487c-a298-9ce8879e5a77 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:45.003797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 676/1354] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:45.009786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1a6b-da2d-4df0-97d1-4f964309c1aa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:45.010233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1a6b-da2d-4df0-97d1-4f964309c1aa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:45.010465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1a6b-da2d-4df0-97d1-4f964309c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:45.087164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bafe1a6b-da2d-4df0-97d1-4f964309c1aa None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:45.088355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bafe1a6b-da2d-4df0-97d1-4f964309c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:45.196799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 679/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:48.619616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798876-34fa-4c47-813f-ed7dc233b633 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:48.620430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798876-34fa-4c47-813f-ed7dc233b633 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:48.620877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f798876-34fa-4c47-813f-ed7dc233b633 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:48.622664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 677/1356] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:48.628240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eefc28-0cdc-4840-8038-213687de8542 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:48.629192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eefc28-0cdc-4840-8038-213687de8542 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:48.629985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eefc28-0cdc-4840-8038-213687de8542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:48.669128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:48.706655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-12eefc28-0cdc-4840-8038-213687de8542 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:48.707990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-12eefc28-0cdc-4840-8038-213687de8542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:48.827612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 680/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:48.883477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f373-4755-4ae7-9ca5-8e1b635d9a3b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:48.883841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f373-4755-4ae7-9ca5-8e1b635d9a3b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:48.884180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f373-4755-4ae7-9ca5-8e1b635d9a3b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:48.886344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 678/1358] 10.222.0.49 () {54 vars in 962 bytes} [Sun Jul 31 18:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:48.893242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad949f4-c0b0-4b81-97ad-4174194be5a2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:48.893837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad949f4-c0b0-4b81-97ad-4174194be5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:48.894306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad949f4-c0b0-4b81-97ad-4174194be5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:48.983327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3ad949f4-c0b0-4b81-97ad-4174194be5a2 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:48.984551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ad949f4-c0b0-4b81-97ad-4174194be5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:49.035580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:49.107540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 681/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Sun Jul 31 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:49.115186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f373-4755-4ae7-9ca5-8e1b635d9a3b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:49.116668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-abc8f373-4755-4ae7-9ca5-8e1b635d9a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:49.268434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f373-4755-4ae7-9ca5-8e1b635d9a3b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:49.351657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:49.393461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:49.394091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:49.394560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:49.394834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:49.396559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:49.490762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:49.492360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbd45e-9efc-48bb-9788-ee0b07c82d91 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:49.512000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 679/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:51.702132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa28e9-10c6-44b4-9461-92aaa77ae458 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:51.702705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa28e9-10c6-44b4-9461-92aaa77ae458 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:51.702705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa28e9-10c6-44b4-9461-92aaa77ae458 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:51.704246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 682/1361] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:51.709281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5953c4-ca8d-43dc-ad6c-07ee0ec6cb51 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:51.709709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5953c4-ca8d-43dc-ad6c-07ee0ec6cb51 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:51.709808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5953c4-ca8d-43dc-ad6c-07ee0ec6cb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:51.778846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bd5953c4-ca8d-43dc-ad6c-07ee0ec6cb51 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:51.779968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bd5953c4-ca8d-43dc-ad6c-07ee0ec6cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:51.819757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:51.894319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 680/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:51.938676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa28e9-10c6-44b4-9461-92aaa77ae458 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:51.940354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-87fa28e9-10c6-44b4-9461-92aaa77ae458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:52.104838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa28e9-10c6-44b4-9461-92aaa77ae458 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:52.120478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:52.235861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:52.236478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:52.236745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:52.237036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:52.238628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:52.333947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:52.335725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88de656-ade7-4ca5-bbf0-b058c229a9c5 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:52.355744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 683/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:54.660301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd2d7e-fa0a-44d2-81f9-f8ac3b4e6745 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:54.660921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd2d7e-fa0a-44d2-81f9-f8ac3b4e6745 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:54.660921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd2d7e-fa0a-44d2-81f9-f8ac3b4e6745 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:54.662400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 681/1364] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:54.667971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb31dee-6604-4084-8b14-60e49bdb63bc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:54.668456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb31dee-6604-4084-8b14-60e49bdb63bc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:54.668552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb31dee-6604-4084-8b14-60e49bdb63bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:54.684631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:54.742750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5cb31dee-6604-4084-8b14-60e49bdb63bc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:54.743962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5cb31dee-6604-4084-8b14-60e49bdb63bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:54.860923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 684/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:54.902455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fe5db-e31e-46f0-8a52-a8b39cce9918 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:54.902704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fe5db-e31e-46f0-8a52-a8b39cce9918 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:54.903136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fe5db-e31e-46f0-8a52-a8b39cce9918 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:54.904901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 682/1366] 10.222.0.49 () {54 vars in 962 bytes} [Sun Jul 31 18:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:54.910533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30faae20-0596-457b-92d9-ea001d8f46fb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:54.910725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30faae20-0596-457b-92d9-ea001d8f46fb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:54.911123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30faae20-0596-457b-92d9-ea001d8f46fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:54.994418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-30faae20-0596-457b-92d9-ea001d8f46fb None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:54.995609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30faae20-0596-457b-92d9-ea001d8f46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:55.034032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:55.118426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 685/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Sun Jul 31 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:55.128179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fe5db-e31e-46f0-8a52-a8b39cce9918 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:55.130258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b93fe5db-e31e-46f0-8a52-a8b39cce9918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:55.297065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fe5db-e31e-46f0-8a52-a8b39cce9918 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:55.318961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:55.418139 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:55.418761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:55.419145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:55.419414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:55.421550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:55.517470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:55.518962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36b387-647b-444b-bc2f-1c670a8d6216 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:55.537920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 683/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:56.479015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939a369-d143-446d-9525-bef3c89a2609 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:56.479582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939a369-d143-446d-9525-bef3c89a2609 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:56.479582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939a369-d143-446d-9525-bef3c89a2609 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:56.481400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 686/1369] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:56.486723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6587a-fbbd-4a27-8fe7-9e3f11b50eed None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:56.487207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6587a-fbbd-4a27-8fe7-9e3f11b50eed None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:56.487444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6587a-fbbd-4a27-8fe7-9e3f11b50eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:56.522090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:56.560513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9fd6587a-fbbd-4a27-8fe7-9e3f11b50eed None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:56.561930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9fd6587a-fbbd-4a27-8fe7-9e3f11b50eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:56.671054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 684/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:29:57.022745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639d9a-7435-4183-bd7f-f8d0cd2af934 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:57.022745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639d9a-7435-4183-bd7f-f8d0cd2af934 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:57.022745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639d9a-7435-4183-bd7f-f8d0cd2af934 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:57.026259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 687/1371] 10.222.0.49 () {54 vars in 964 bytes} [Sun Jul 31 18:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:57.029151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247e1ee-8cb4-4581-aa1e-98a957830782 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:57.029499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247e1ee-8cb4-4581-aa1e-98a957830782 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:57.029834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247e1ee-8cb4-4581-aa1e-98a957830782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:57.114653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3247e1ee-8cb4-4581-aa1e-98a957830782 None None] MFA Rules not processed for user `20341bc6269f477c952c25437ab7a2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:57.115976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3247e1ee-8cb4-4581-aa1e-98a957830782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:57.165832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:57.200923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 685/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Sun Jul 31 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:57.206196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639d9a-7435-4183-bd7f-f8d0cd2af934 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:57.208120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa639d9a-7435-4183-bd7f-f8d0cd2af934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:57.325964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639d9a-7435-4183-bd7f-f8d0cd2af934 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:57.369097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:57.404909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:57.405711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:57.406141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:57.406528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:57.408332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:57.525636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:57.527102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d40701-c3ae-4b5f-8a05-d1e083f4307d service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:57.547926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 688/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:29:57.636159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151c62-a41c-431f-95d0-b3037ea99f3c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:57.636613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151c62-a41c-431f-95d0-b3037ea99f3c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:57.636863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151c62-a41c-431f-95d0-b3037ea99f3c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:57.638802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 686/1374] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:29:57.646236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a010a-fd60-4cd1-9a3e-f6ea4ffbbe29 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:57.646236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a010a-fd60-4cd1-9a3e-f6ea4ffbbe29 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:57.646236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a010a-fd60-4cd1-9a3e-f6ea4ffbbe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:57.715263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-dc8a010a-fd60-4cd1-9a3e-f6ea4ffbbe29 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:29:57.716411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dc8a010a-fd60-4cd1-9a3e-f6ea4ffbbe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:57.737856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:57.819511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 689/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:29:57.862456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62151c62-a41c-431f-95d0-b3037ea99f3c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:29:57.863881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-62151c62-a41c-431f-95d0-b3037ea99f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:58.021829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62151c62-a41c-431f-95d0-b3037ea99f3c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:29:58.075910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:29:58.140119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:29:58.140795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:29:58.141109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:29:58.141377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:29:58.142991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:29:58.234733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:29:58.236351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3e63d-26f0-40ec-bd5b-f53e866df8da service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:29:58.255895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 687/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:00.258672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f491a0-3090-4953-a215-084216d88e93 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:00.259173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f491a0-3090-4953-a215-084216d88e93 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:00.259173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f491a0-3090-4953-a215-084216d88e93 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:00.260587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 690/1377] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:00.265880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ed30-b94d-4a92-9123-11a8e35af8ab None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:00.266371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ed30-b94d-4a92-9123-11a8e35af8ab None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:00.266371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ed30-b94d-4a92-9123-11a8e35af8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:00.335604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7de4ed30-b94d-4a92-9123-11a8e35af8ab None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:00.342176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7de4ed30-b94d-4a92-9123-11a8e35af8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:00.347993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:00.481226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 688/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:00.519636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f491a0-3090-4953-a215-084216d88e93 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:00.525349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-16f491a0-3090-4953-a215-084216d88e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:00.683334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f491a0-3090-4953-a215-084216d88e93 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:00.686865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:00.806950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:00.807486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:00.807762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:00.808030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:00.809714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:00.921275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:00.923584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9906b-247d-48ff-abe2-a9bda8df775d service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:00.967142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 691/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:03.071465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb8bfb-a51a-4f17-9d2d-48337bd4abdd None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:03.072005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb8bfb-a51a-4f17-9d2d-48337bd4abdd None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:03.072185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb8bfb-a51a-4f17-9d2d-48337bd4abdd None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:03.074167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 689/1380] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:03.080111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d030-16c1-43a8-acf6-c41aedbe3216 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:03.080567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d030-16c1-43a8-acf6-c41aedbe3216 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:03.080647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d030-16c1-43a8-acf6-c41aedbe3216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:03.092367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:03.156478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1932d030-16c1-43a8-acf6-c41aedbe3216 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:03.157738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1932d030-16c1-43a8-acf6-c41aedbe3216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:03.266485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 692/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:03.309683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb8bfb-a51a-4f17-9d2d-48337bd4abdd None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:03.311163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-14fb8bfb-a51a-4f17-9d2d-48337bd4abdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:03.458952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:03.471750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb8bfb-a51a-4f17-9d2d-48337bd4abdd None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:03.589046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:03.589578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:03.589848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:03.590119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:03.591728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:03.684587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:03.686409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7130a-0fa3-441b-be8e-0a4c1712c4f1 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:03.697800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:03.716231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 690/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:05.731675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e195f64-be48-4616-9dc9-d26061bfee5a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:05.732424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e195f64-be48-4616-9dc9-d26061bfee5a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:05.732424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e195f64-be48-4616-9dc9-d26061bfee5a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:05.733898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 693/1383] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:05.739807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bff3d4-485d-4b1b-a5da-740dc2b745e9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:05.740294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bff3d4-485d-4b1b-a5da-740dc2b745e9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:05.740534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bff3d4-485d-4b1b-a5da-740dc2b745e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:05.814297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-46bff3d4-485d-4b1b-a5da-740dc2b745e9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:05.815994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-46bff3d4-485d-4b1b-a5da-740dc2b745e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:05.934700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 691/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:05.982173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e195f64-be48-4616-9dc9-d26061bfee5a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:05.983681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1e195f64-be48-4616-9dc9-d26061bfee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:06.078475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:06.143480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e195f64-be48-4616-9dc9-d26061bfee5a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:06.289226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:06.289937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:06.290279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:06.290613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:06.292360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:06.358356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:06.419806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:06.421638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdce9ee-ad50-4d30-a9d3-a78000d46c7c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:06.440616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 694/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:08.481049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b746a9-84c1-4810-8c40-0837be843eee None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:08.481619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b746a9-84c1-4810-8c40-0837be843eee None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:08.481733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b746a9-84c1-4810-8c40-0837be843eee None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:08.483108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 692/1386] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:08.488818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500945bd-d1c3-4fd5-aac3-8af97cbb0e5d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:08.489391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500945bd-d1c3-4fd5-aac3-8af97cbb0e5d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:08.489391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500945bd-d1c3-4fd5-aac3-8af97cbb0e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:08.562680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-500945bd-d1c3-4fd5-aac3-8af97cbb0e5d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:08.564023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-500945bd-d1c3-4fd5-aac3-8af97cbb0e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:08.681274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 695/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:08.750744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b746a9-84c1-4810-8c40-0837be843eee None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:08.753080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-58b746a9-84c1-4810-8c40-0837be843eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:08.844099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:08.922278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b746a9-84c1-4810-8c40-0837be843eee None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:09.060202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:09.061000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:09.061299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:09.061547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:09.063273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:09.107174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:09.172261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:09.173968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b6ac8-19ee-48ea-a1b6-4ab76a579d86 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:09.193602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 693/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:11.327107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe840755-e3bd-4838-9b1d-e3ffb7725bdf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:11.327740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe840755-e3bd-4838-9b1d-e3ffb7725bdf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:11.327740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe840755-e3bd-4838-9b1d-e3ffb7725bdf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:11.330578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 696/1389] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:11.336171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598e86-0c4d-4c64-b236-dd1cffe0b461 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:11.336637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598e86-0c4d-4c64-b236-dd1cffe0b461 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:11.336739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598e86-0c4d-4c64-b236-dd1cffe0b461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:11.405935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-48598e86-0c4d-4c64-b236-dd1cffe0b461 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:11.407142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-48598e86-0c4d-4c64-b236-dd1cffe0b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:11.515033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 694/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:11.560687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe840755-e3bd-4838-9b1d-e3ffb7725bdf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:11.562349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fe840755-e3bd-4838-9b1d-e3ffb7725bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:11.596811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:11.725475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe840755-e3bd-4838-9b1d-e3ffb7725bdf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:11.853338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:11.870928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:11.871555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:11.871827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:11.872091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:11.873711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:11.972815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:11.974560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b50446-7606-4bc3-94b0-288d076fec7f service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:11.994335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 697/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:12.542165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb5bc-bf57-4471-8feb-53ebc681f093 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:12.542706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb5bc-bf57-4471-8feb-53ebc681f093 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:12.542706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb5bc-bf57-4471-8feb-53ebc681f093 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:12.544071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 695/1392] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:12.549563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1df5-b9e1-4756-ac65-be9b707965b7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:12.549968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1df5-b9e1-4756-ac65-be9b707965b7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:12.550077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1df5-b9e1-4756-ac65-be9b707965b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:12.614876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-415b1df5-b9e1-4756-ac65-be9b707965b7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:12.616807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-415b1df5-b9e1-4756-ac65-be9b707965b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:12.675384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:12.714390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 698/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:12.772253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4040547-bd59-4459-af16-286560afb75d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:12.772741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4040547-bd59-4459-af16-286560afb75d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:12.773211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4040547-bd59-4459-af16-286560afb75d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:12.775060 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 696/1394] 10.222.0.49 () {54 vars in 966 bytes} [Sun Jul 31 18:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:12.780101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e166d18-a4cd-48f2-bc6e-4a7097d99bfa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:12.780391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e166d18-a4cd-48f2-bc6e-4a7097d99bfa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:12.780712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e166d18-a4cd-48f2-bc6e-4a7097d99bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:12.860168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2e166d18-a4cd-48f2-bc6e-4a7097d99bfa None None] MFA Rules not processed for user `7e84fa2255614cde84495eedcc0971a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:12.861678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e166d18-a4cd-48f2-bc6e-4a7097d99bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:12.938371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 699/1395] 10.222.0.49 () {58 vars in 1063 bytes} [Sun Jul 31 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:12.946312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4040547-bd59-4459-af16-286560afb75d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:12.947897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f4040547-bd59-4459-af16-286560afb75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:12.962365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:13.064233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4040547-bd59-4459-af16-286560afb75d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:13.140147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:13.140811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:13.141092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:13.141357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:13.143045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:13.213288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:13.258062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:13.259561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc9d49-1cbd-4e52-986e-a6c3b486a595 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:13.279086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 697/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:14.012919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ec65-e6a5-4fa0-b22e-161719a723ec None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.013415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ec65-e6a5-4fa0-b22e-161719a723ec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.013517 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ec65-e6a5-4fa0-b22e-161719a723ec None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.014857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 700/1397] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:14.017408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b070cb-286e-4d42-bb40-c9def906ef4d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.017796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b070cb-286e-4d42-bb40-c9def906ef4d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.017977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b070cb-286e-4d42-bb40-c9def906ef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.020230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7af0b-20eb-4176-803e-0c93f7597236 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.020514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7af0b-20eb-4176-803e-0c93f7597236 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.020859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7af0b-20eb-4176-803e-0c93f7597236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.084904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-13b070cb-286e-4d42-bb40-c9def906ef4d None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:14.086084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-13b070cb-286e-4d42-bb40-c9def906ef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:14.089076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f9d7af0b-20eb-4176-803e-0c93f7597236 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:14.090261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f9d7af0b-20eb-4176-803e-0c93f7597236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:14.098689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:14.176996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 698/1398] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:14.190848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 701/1399] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:14.221630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c5fe-b24c-4c46-bbe2-8fd5950f5149 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.221630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c5fe-b24c-4c46-bbe2-8fd5950f5149 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.221630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c5fe-b24c-4c46-bbe2-8fd5950f5149 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.222743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 699/1400] 10.222.0.49 () {54 vars in 943 bytes} [Sun Jul 31 18:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:14.230036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.230316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.230590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.235520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238c5fe-b24c-4c46-bbe2-8fd5950f5149 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:14.237105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1238c5fe-b24c-4c46-bbe2-8fd5950f5149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:14.273837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:14.309050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] MFA Rules not processed for user `e64871a43e564c9bbf7ebfdbd512a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:14.310316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:14.388781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:14.391463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238c5fe-b24c-4c46-bbe2-8fd5950f5149 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:14.428542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 702/1401] 10.222.0.49 () {58 vars in 1040 bytes} [Sun Jul 31 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:14.441658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:14.451402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b511be9-a515-4a1b-9489-8b01a6d2946a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:14.502335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:14.508611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:14.509388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.509660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.509927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.511541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:14.556720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:14.557301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:14.557571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:14.557834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:14.559464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:14.601156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:14.602750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699200f-2fd9-4feb-87f8-e836b2c0ab9f service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:14.621369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 700/1402] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:14.641327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:14.643240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368422c-23de-47a8-9abc-1818888fedc9 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:14.646983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:14.656634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 703/1403] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:30:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:30:16.663717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e7b9b-b8b9-4358-aeb9-6d4161d83475 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:16.664284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e7b9b-b8b9-4358-aeb9-6d4161d83475 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:16.664284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e7b9b-b8b9-4358-aeb9-6d4161d83475 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:16.665671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 701/1404] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:16.671379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d0a34-577b-46f2-a3b2-36ae2e193fdd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:16.671379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d0a34-577b-46f2-a3b2-36ae2e193fdd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:16.671844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d0a34-577b-46f2-a3b2-36ae2e193fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:16.747309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ca7d0a34-577b-46f2-a3b2-36ae2e193fdd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:16.749610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ca7d0a34-577b-46f2-a3b2-36ae2e193fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:16.864848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 704/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:16.908389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e7b9b-b8b9-4358-aeb9-6d4161d83475 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:16.910070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a66e7b9b-b8b9-4358-aeb9-6d4161d83475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:17.037806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:17.075253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e7b9b-b8b9-4358-aeb9-6d4161d83475 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:17.204212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:17.204971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:17.205314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:17.205656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:17.207289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:17.295456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:17.306732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:17.308416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77209e3d-effb-4363-9e4b-929c7ee71216 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:17.332349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 702/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:19.444256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93290b76-9236-461b-af88-72d918c737fa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:19.444896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93290b76-9236-461b-af88-72d918c737fa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:19.444896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93290b76-9236-461b-af88-72d918c737fa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:19.446174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 705/1407] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:19.451381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ecf86-7817-412d-a424-5093439abddc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:19.451831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ecf86-7817-412d-a424-5093439abddc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:19.451939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ecf86-7817-412d-a424-5093439abddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:19.525312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bc7ecf86-7817-412d-a424-5093439abddc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:19.526438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bc7ecf86-7817-412d-a424-5093439abddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:19.639625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 703/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:19.687588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93290b76-9236-461b-af88-72d918c737fa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:19.689147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-93290b76-9236-461b-af88-72d918c737fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:19.778039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:19.851312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93290b76-9236-461b-af88-72d918c737fa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:19.978993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:19.979706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:19.979984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:19.980264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:19.981891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:20.025768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:20.080454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:20.082017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026c149-0cf5-4633-b81e-44301713628d service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:20.101443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 706/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:29.637549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=74 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:30.098218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=28 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:36.586833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29e102-d86f-433d-8344-149073583e44 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:36.587433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29e102-d86f-433d-8344-149073583e44 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:36.587433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29e102-d86f-433d-8344-149073583e44 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:36.588714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 704/1410] 10.222.0.49 () {54 vars in 918 bytes} [Sun Jul 31 18:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:36.594233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485529c3-2d24-46e7-9cde-57949b850b1d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:36.594676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485529c3-2d24-46e7-9cde-57949b850b1d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:36.594777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485529c3-2d24-46e7-9cde-57949b850b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:36.682582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-485529c3-2d24-46e7-9cde-57949b850b1d None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:36.684408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-485529c3-2d24-46e7-9cde-57949b850b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:36.800539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 707/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Sun Jul 31 18:30:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:36.815726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88df663-cc5e-470a-b1b2-6f53d389cde6 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:36.816033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88df663-cc5e-470a-b1b2-6f53d389cde6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:36.816306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88df663-cc5e-470a-b1b2-6f53d389cde6 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:36.817693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 705/1412] 10.222.0.49 () {54 vars in 918 bytes} [Sun Jul 31 18:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:36.822617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7e6f-66b1-488d-af08-fb5108850896 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:36.822905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7e6f-66b1-488d-af08-fb5108850896 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:36.823170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7e6f-66b1-488d-af08-fb5108850896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:36.905172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:36.908595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5a2c7e6f-66b1-488d-af08-fb5108850896 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:36.910659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5a2c7e6f-66b1-488d-af08-fb5108850896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:37.032774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 708/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Sun Jul 31 18:30:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:39.503200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03774-7422-41d1-ac90-a1887dee47a5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:39.504071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03774-7422-41d1-ac90-a1887dee47a5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:39.504071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03774-7422-41d1-ac90-a1887dee47a5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:39.506762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:30:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:39.513203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d59b-baf3-4fb3-8914-8f2bc1d3182c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:39.513662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d59b-baf3-4fb3-8914-8f2bc1d3182c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:39.513757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d59b-baf3-4fb3-8914-8f2bc1d3182c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:39.583704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9327d59b-baf3-4fb3-8914-8f2bc1d3182c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:39.584953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9327d59b-baf3-4fb3-8914-8f2bc1d3182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:39.641541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:39.683125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:39.735748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03774-7422-41d1-ac90-a1887dee47a5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:39.737369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-84a03774-7422-41d1-ac90-a1887dee47a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:39.891580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03774-7422-41d1-ac90-a1887dee47a5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:39.972271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:40.014631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:40.015242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:40.015512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:40.015776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:40.017405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:40.112407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:40.114047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7b944-fcc2-48b1-8fe1-4f7bf7f09520 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:40.135944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 707/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:41.039088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83440-662a-4055-9e83-88988bf34919 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:41.039923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83440-662a-4055-9e83-88988bf34919 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:41.039923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83440-662a-4055-9e83-88988bf34919 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:41.042021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:41.049111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daffeba-374f-49b8-8896-d656dd09b992 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:41.049579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daffeba-374f-49b8-8896-d656dd09b992 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:41.049681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daffeba-374f-49b8-8896-d656dd09b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:41.128974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9daffeba-374f-49b8-8896-d656dd09b992 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:41.130557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9daffeba-374f-49b8-8896-d656dd09b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:41.161250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:41.232704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:30:41.381549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83440-662a-4055-9e83-88988bf34919 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:41.383737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fff83440-662a-4055-9e83-88988bf34919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:41.676931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83440-662a-4055-9e83-88988bf34919 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:41.728301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:41.807768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:41.808394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:41.808659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:41.808961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:41.810642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:41.901873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:41.903453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c1b90-8fed-4c9c-b0ba-b4a4f537de26 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:41.921525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 711/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:42.803855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943e093-d6c6-45dd-b01f-c8ca5ddbeed8 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:42.804473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943e093-d6c6-45dd-b01f-c8ca5ddbeed8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:42.804473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943e093-d6c6-45dd-b01f-c8ca5ddbeed8 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:42.805823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:42.811851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242eb4b8-e504-4fc6-ac90-f936b3834910 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:42.812474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242eb4b8-e504-4fc6-ac90-f936b3834910 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:42.812592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242eb4b8-e504-4fc6-ac90-f936b3834910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:42.868204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:42.892665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-242eb4b8-e504-4fc6-ac90-f936b3834910 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:42.893937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-242eb4b8-e504-4fc6-ac90-f936b3834910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:42.995864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:43.161534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943e093-d6c6-45dd-b01f-c8ca5ddbeed8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:43.163906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4943e093-d6c6-45dd-b01f-c8ca5ddbeed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:43.321546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943e093-d6c6-45dd-b01f-c8ca5ddbeed8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:43.334659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:43.453958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:43.454484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:43.454762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:43.455024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:43.456642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:43.553452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:43.555039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efc93-e8a9-44d7-99d2-9b15ddfaf99d service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:43.574025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 710/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:44.420267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ae724-4341-4488-b6d7-c7e75d7e2934 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:44.420785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ae724-4341-4488-b6d7-c7e75d7e2934 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:44.420785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ae724-4341-4488-b6d7-c7e75d7e2934 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:44.422196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:44.428222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074923d-8bce-4342-ba0f-4c24a57d4f53 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:44.428665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074923d-8bce-4342-ba0f-4c24a57d4f53 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:44.428846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074923d-8bce-4342-ba0f-4c24a57d4f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:44.452977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:44.513300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3074923d-8bce-4342-ba0f-4c24a57d4f53 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:44.514421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3074923d-8bce-4342-ba0f-4c24a57d4f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:44.618222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:44.769350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ae724-4341-4488-b6d7-c7e75d7e2934 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:44.770774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a04ae724-4341-4488-b6d7-c7e75d7e2934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:44.892862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:44.928958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ae724-4341-4488-b6d7-c7e75d7e2934 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:45.043955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:45.044485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:45.044780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:45.045058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:45.046687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:45.126722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:45.148997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:45.151394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02714eb8-fb76-4840-87bd-2acc86b090f7 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:45.172214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 714/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:46.019844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62c06-548c-441d-89e3-bf1694b94c8b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:46.020618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62c06-548c-441d-89e3-bf1694b94c8b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:46.021005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62c06-548c-441d-89e3-bf1694b94c8b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:46.022968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:46.030420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcff095-847b-4d4d-999e-60bbb6af1201 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:46.030860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcff095-847b-4d4d-999e-60bbb6af1201 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:46.030945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcff095-847b-4d4d-999e-60bbb6af1201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:46.109238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-edcff095-847b-4d4d-999e-60bbb6af1201 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:46.110458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-edcff095-847b-4d4d-999e-60bbb6af1201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:46.211737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:46.360684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62c06-548c-441d-89e3-bf1694b94c8b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:46.362172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-86a62c06-548c-441d-89e3-bf1694b94c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:46.453489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:46.527543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62c06-548c-441d-89e3-bf1694b94c8b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:46.652274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:46.652922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:46.653202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:46.653471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:46.655120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:46.713367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:46.764913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:46.766500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49138d72-a12b-4ec6-a4fd-3a0a3e6f4007 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:46.785590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 713/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:47.599211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b0b77-688a-4d8c-9d80-e251021a1d16 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:47.599774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b0b77-688a-4d8c-9d80-e251021a1d16 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:47.599774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b0b77-688a-4d8c-9d80-e251021a1d16 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:47.601088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:47.607182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c375ba3-07d5-4ee9-af8a-c5b7a0981742 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:47.607641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c375ba3-07d5-4ee9-af8a-c5b7a0981742 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:47.607738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c375ba3-07d5-4ee9-af8a-c5b7a0981742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:47.698632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1c375ba3-07d5-4ee9-af8a-c5b7a0981742 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:47.700461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1c375ba3-07d5-4ee9-af8a-c5b7a0981742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:47.820940 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:47.978072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b0b77-688a-4d8c-9d80-e251021a1d16 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:47.980546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d83b0b77-688a-4d8c-9d80-e251021a1d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:48.035274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:48.140223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b0b77-688a-4d8c-9d80-e251021a1d16 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:48.263660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:48.264190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:48.264457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:48.264714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:48.266361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:48.273397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:48.358766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:48.360423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764c047-7766-4eef-9d06-f9c2e61a0450 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:48.379689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 717/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:49.184586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82142fd-c10d-4305-adf1-b59e6b64a0de None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:49.185221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82142fd-c10d-4305-adf1-b59e6b64a0de None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:49.185221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82142fd-c10d-4305-adf1-b59e6b64a0de None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:49.187295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:49.193541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a9c9-3f6b-4e53-aa4d-91de3d7a31ca None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:49.193998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a9c9-3f6b-4e53-aa4d-91de3d7a31ca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:49.194100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a9c9-3f6b-4e53-aa4d-91de3d7a31ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:49.296543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-64f8a9c9-3f6b-4e53-aa4d-91de3d7a31ca None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:49.297932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-64f8a9c9-3f6b-4e53-aa4d-91de3d7a31ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:49.370563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:49.400611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:49.552304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82142fd-c10d-4305-adf1-b59e6b64a0de None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:49.554740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b82142fd-c10d-4305-adf1-b59e6b64a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:49.575500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:49.718749 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82142fd-c10d-4305-adf1-b59e6b64a0de None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:49.813619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:49.843057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:49.843678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:49.845927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:49.846240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:49.847872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:49.943962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:49.945622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eab4f-34c8-4954-9429-4762d3fd6046 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:49.964136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 716/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:50.765297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947923-1450-4434-b038-04e3739edf74 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:50.766207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947923-1450-4434-b038-04e3739edf74 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:50.766207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947923-1450-4434-b038-04e3739edf74 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:50.767595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:50.772694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3784444-6b29-4ca1-8fd1-37f9bcf9a06e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:50.773149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3784444-6b29-4ca1-8fd1-37f9bcf9a06e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:50.773285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3784444-6b29-4ca1-8fd1-37f9bcf9a06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:50.855667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e3784444-6b29-4ca1-8fd1-37f9bcf9a06e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:50.857565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e3784444-6b29-4ca1-8fd1-37f9bcf9a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:50.905054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:50.961288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:51.113375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947923-1450-4434-b038-04e3739edf74 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:51.114843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-11947923-1450-4434-b038-04e3739edf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:51.274269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947923-1450-4434-b038-04e3739edf74 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:51.329218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:51.392719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:51.393382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:51.393657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:51.393919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:51.395543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:51.484126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:51.485796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62010e-7db7-4356-a189-a97ec0527e10 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:51.503853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 720/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:30:52.317531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457bf1e-c647-4ed7-9cf8-31c67fb2ae44 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:52.318112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457bf1e-c647-4ed7-9cf8-31c67fb2ae44 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:52.318112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457bf1e-c647-4ed7-9cf8-31c67fb2ae44 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:52.319446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:52.327905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8c0c-e5f1-4ce8-abe3-efa11fc2a00b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:52.328590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8c0c-e5f1-4ce8-abe3-efa11fc2a00b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:52.328780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8c0c-e5f1-4ce8-abe3-efa11fc2a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:52.409797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-566f8c0c-e5f1-4ce8-abe3-efa11fc2a00b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:52.411016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-566f8c0c-e5f1-4ce8-abe3-efa11fc2a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:52.413061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:52.512991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:52.663017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457bf1e-c647-4ed7-9cf8-31c67fb2ae44 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:52.665409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c457bf1e-c647-4ed7-9cf8-31c67fb2ae44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:52.835544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457bf1e-c647-4ed7-9cf8-31c67fb2ae44 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:52.867063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:52.958377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:52.958996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:52.959265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:52.959527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:52.961207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:53.064978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:53.066577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da77e6-e2cf-4086-8ee4-5e508dc49206 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:53.086025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 719/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:54.993897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07a6b-083c-4395-a6d9-dc335f646e5b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:54.994456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07a6b-083c-4395-a6d9-dc335f646e5b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:54.994456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07a6b-083c-4395-a6d9-dc335f646e5b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:54.995788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:55.001252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68164d7-0955-4a26-91ee-cdc5e867b665 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:55.001721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68164d7-0955-4a26-91ee-cdc5e867b665 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:55.001833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68164d7-0955-4a26-91ee-cdc5e867b665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:55.047054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:55.074464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b68164d7-0955-4a26-91ee-cdc5e867b665 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:55.075584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b68164d7-0955-4a26-91ee-cdc5e867b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:55.179654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:55.226766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d07a6b-083c-4395-a6d9-dc335f646e5b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:55.228297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8d07a6b-083c-4395-a6d9-dc335f646e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:55.345956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d07a6b-083c-4395-a6d9-dc335f646e5b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:55.378485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:55.423679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:55.424299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:55.424568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:55.424892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:55.426545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:55.543392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:55.544874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8ea8f-8674-4107-953d-04e32ab54ed9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:55.564074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 723/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:30:57.715971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68f085-7c6c-4347-8af9-5859ffedf8de None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:57.716568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68f085-7c6c-4347-8af9-5859ffedf8de None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:57.716568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68f085-7c6c-4347-8af9-5859ffedf8de None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:57.717892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:30:57.723995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684b10-62e7-4bcb-838e-d06a9fc69927 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:57.724717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684b10-62e7-4bcb-838e-d06a9fc69927 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:57.725023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684b10-62e7-4bcb-838e-d06a9fc69927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:57.797425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-57684b10-62e7-4bcb-838e-d06a9fc69927 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:30:57.799083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-57684b10-62e7-4bcb-838e-d06a9fc69927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:57.812045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:57.905331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:30:57.946885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68f085-7c6c-4347-8af9-5859ffedf8de None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:30:57.948435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3a68f085-7c6c-4347-8af9-5859ffedf8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:58.062946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68f085-7c6c-4347-8af9-5859ffedf8de None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:30:58.075664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:30:58.144792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:30:58.145432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:30:58.145696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:30:58.145961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:30:58.147616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:30:58.260343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:30:58.261893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4c9bf-0837-442e-8fd5-bc837326cc74 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:30:58.281397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 722/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:00.587822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fa045-903f-45e4-b3e3-28b18980b704 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:00.588400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fa045-903f-45e4-b3e3-28b18980b704 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:00.588400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fa045-903f-45e4-b3e3-28b18980b704 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:00.589761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:00.595172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbf55-98f5-4f53-b976-6383881239db None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:00.595671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbf55-98f5-4f53-b976-6383881239db None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:00.595777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbf55-98f5-4f53-b976-6383881239db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:00.645244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:00.667842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e8fbf55-98f5-4f53-b976-6383881239db None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:00.669085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e8fbf55-98f5-4f53-b976-6383881239db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:00.775016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:31:00.815868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fa045-903f-45e4-b3e3-28b18980b704 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:00.817418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-173fa045-903f-45e4-b3e3-28b18980b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:00.922627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fa045-903f-45e4-b3e3-28b18980b704 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:00.963317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:00.997716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:00.998343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:00.998619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:00.998887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:01.000589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:01.130699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:01.132143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc09c7f-19f1-4b7a-9b14-e2126782892a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:01.150678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 726/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:31:03.362831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70337cc-7771-4bf1-8296-d7d4501d2a86 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:03.363395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70337cc-7771-4bf1-8296-d7d4501d2a86 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:03.363395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70337cc-7771-4bf1-8296-d7d4501d2a86 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:03.364723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:03.371041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434c4fd-71b7-4e77-8d63-2b1152776823 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:03.371657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434c4fd-71b7-4e77-8d63-2b1152776823 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:03.371767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434c4fd-71b7-4e77-8d63-2b1152776823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:03.445998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3434c4fd-71b7-4e77-8d63-2b1152776823 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:03.447229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3434c4fd-71b7-4e77-8d63-2b1152776823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:03.472408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:03.558268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:03.599572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70337cc-7771-4bf1-8296-d7d4501d2a86 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:03.601141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a70337cc-7771-4bf1-8296-d7d4501d2a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:03.717701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70337cc-7771-4bf1-8296-d7d4501d2a86 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:03.738981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:03.796901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:03.797528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:03.797809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:03.798081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:03.799711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:03.914791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:03.916319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7e1df-e23c-4774-8e2c-68b1bb235dd5 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:03.939679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 725/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:05.989434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876834f3-3a7c-4eb7-b091-291763cf271a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:05.990051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876834f3-3a7c-4eb7-b091-291763cf271a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:05.990051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876834f3-3a7c-4eb7-b091-291763cf271a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:05.991332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:05.997857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d59d-8274-44be-9069-5149a7cc2e03 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:05.998595 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d59d-8274-44be-9069-5149a7cc2e03 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:05.998724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d59d-8274-44be-9069-5149a7cc2e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:06.071101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:06.074543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9a19d59d-8274-44be-9069-5149a7cc2e03 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:06.076453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9a19d59d-8274-44be-9069-5149a7cc2e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:06.186728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:06.230706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876834f3-3a7c-4eb7-b091-291763cf271a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:06.233452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-876834f3-3a7c-4eb7-b091-291763cf271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:06.355766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876834f3-3a7c-4eb7-b091-291763cf271a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:06.411407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:06.436833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:06.437498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:06.437772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:06.438035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:06.439659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:06.554585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:06.556019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dc9a1-e8f5-42b5-861c-e3b3893a6fe4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:06.575376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 729/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:08.601511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d99ec-05b9-4192-8858-99075abe9d54 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:08.602353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d99ec-05b9-4192-8858-99075abe9d54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:08.602353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d99ec-05b9-4192-8858-99075abe9d54 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:08.604394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:08.610114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464019-5ed1-4c43-b79d-32f359e52c92 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:08.610580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464019-5ed1-4c43-b79d-32f359e52c92 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:08.610699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464019-5ed1-4c43-b79d-32f359e52c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:08.681305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f6464019-5ed1-4c43-b79d-32f359e52c92 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:08.682546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6464019-5ed1-4c43-b79d-32f359e52c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:08.715323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:08.791796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:31:08.829418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d99ec-05b9-4192-8858-99075abe9d54 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:08.830947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c47d99ec-05b9-4192-8858-99075abe9d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:08.952955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d99ec-05b9-4192-8858-99075abe9d54 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:08.987251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:09.048012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:09.048625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:09.048943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:09.049221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:09.050861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:09.179800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:09.181312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d266eb-8bda-43c2-b507-305b02af28d9 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:09.200618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 728/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:11.283038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403bb5a-0fb3-4320-bd80-797c3d624130 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:11.283852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403bb5a-0fb3-4320-bd80-797c3d624130 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:11.284078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403bb5a-0fb3-4320-bd80-797c3d624130 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:11.286240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:11.292541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a987491-419f-4788-b079-6c908587a662 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:11.293302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a987491-419f-4788-b079-6c908587a662 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:11.293474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a987491-419f-4788-b079-6c908587a662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:11.367269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4a987491-419f-4788-b079-6c908587a662 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:11.368481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4a987491-419f-4788-b079-6c908587a662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:11.377039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:11.470078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:11.511574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403bb5a-0fb3-4320-bd80-797c3d624130 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:11.514031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5403bb5a-0fb3-4320-bd80-797c3d624130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:11.650076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403bb5a-0fb3-4320-bd80-797c3d624130 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:11.714133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:11.733545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:11.734257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:11.734531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:11.734791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:11.736413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:11.855524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:11.856930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a3c12-624a-409e-a10f-9631acbefb55 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:11.876035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 732/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:13.975983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f1083-5243-4024-a499-f1a5d5f3c5d1 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:13.976588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f1083-5243-4024-a499-f1a5d5f3c5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:13.976588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f1083-5243-4024-a499-f1a5d5f3c5d1 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:13.978024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:13.984471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970e5d2-0c16-4e60-9739-1721f58b0b57 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:13.984946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970e5d2-0c16-4e60-9739-1721f58b0b57 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:13.985041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970e5d2-0c16-4e60-9739-1721f58b0b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:14.074459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2970e5d2-0c16-4e60-9739-1721f58b0b57 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:14.075658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2970e5d2-0c16-4e60-9739-1721f58b0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:14.114838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:14.177742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:14.224990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f1083-5243-4024-a499-f1a5d5f3c5d1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:14.226668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a84f1083-5243-4024-a499-f1a5d5f3c5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:14.344330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f1083-5243-4024-a499-f1a5d5f3c5d1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:14.385073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:14.423454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:14.424081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:14.424358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:14.424631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:14.426294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:14.550363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:14.551708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437c2f4-089f-4a2a-ba9e-587d4d1acc3f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:14.571141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 731/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:16.612296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c805-f383-4e32-b41f-5d5dfd5ad33b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:16.613178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c805-f383-4e32-b41f-5d5dfd5ad33b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:16.613178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502c805-f383-4e32-b41f-5d5dfd5ad33b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:16.615277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:16.624103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72661a-a643-4582-9b18-8ab829cb639f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:16.624582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72661a-a643-4582-9b18-8ab829cb639f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:16.624789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72661a-a643-4582-9b18-8ab829cb639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:16.695561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ef72661a-a643-4582-9b18-8ab829cb639f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:16.696807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ef72661a-a643-4582-9b18-8ab829cb639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:16.715876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:16.805468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:16.842597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c805-f383-4e32-b41f-5d5dfd5ad33b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:16.844267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9502c805-f383-4e32-b41f-5d5dfd5ad33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:16.953948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502c805-f383-4e32-b41f-5d5dfd5ad33b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:17.026133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:17.033381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:17.033998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:17.034262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:17.034530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:17.036145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:17.146611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:17.148050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a8cf3-e681-45f2-b41f-4a2fafd83ae9 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:17.166506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 735/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:19.161379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde63196-2fd4-4e43-8a00-7a1161db4db5 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:19.162296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde63196-2fd4-4e43-8a00-7a1161db4db5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:19.162296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde63196-2fd4-4e43-8a00-7a1161db4db5 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:19.164406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:31:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:19.172988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd896fe-9b21-4648-806e-0078134223e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:19.173414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd896fe-9b21-4648-806e-0078134223e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:19.173533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd896fe-9b21-4648-806e-0078134223e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:19.243841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-efd896fe-9b21-4648-806e-0078134223e9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:19.245084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-efd896fe-9b21-4648-806e-0078134223e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:19.295745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:19.347925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:19.387962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde63196-2fd4-4e43-8a00-7a1161db4db5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:19.390522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cde63196-2fd4-4e43-8a00-7a1161db4db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:19.501425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde63196-2fd4-4e43-8a00-7a1161db4db5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:19.549118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:19.576782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:19.577432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:19.577708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:19.577974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:19.580863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:19.694296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:19.695828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27873beb-4831-462c-b830-d25c8ef8b361 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:19.720002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 734/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:29.346042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=22 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:29.716506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=66 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:39.095423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf61a0-5fbd-4320-9991-fb9989884de9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:39.096334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf61a0-5fbd-4320-9991-fb9989884de9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:39.096334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf61a0-5fbd-4320-9991-fb9989884de9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:39.098511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 737/1471] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:39.103942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e1f92-2e6d-4168-a32c-69c25a42b2aa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:39.104383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e1f92-2e6d-4168-a32c-69c25a42b2aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:39.104487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e1f92-2e6d-4168-a32c-69c25a42b2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:39.181939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b80e1f92-2e6d-4168-a32c-69c25a42b2aa None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:39.183132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b80e1f92-2e6d-4168-a32c-69c25a42b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:39.298356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 735/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:39.383533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b62cec-ee4c-4381-8b96-5c008d885b13 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:39.383812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b62cec-ee4c-4381-8b96-5c008d885b13 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:39.384074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b62cec-ee4c-4381-8b96-5c008d885b13 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:39.385482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 738/1473] 10.222.0.49 () {54 vars in 966 bytes} [Sun Jul 31 18:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:39.390567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180fd78-28b2-4460-ade3-21364f6c1c54 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:39.390846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180fd78-28b2-4460-ade3-21364f6c1c54 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:39.391109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180fd78-28b2-4460-ade3-21364f6c1c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:39.482701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a180fd78-28b2-4460-ade3-21364f6c1c54 None None] MFA Rules not processed for user `7e84fa2255614cde84495eedcc0971a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:39.484051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a180fd78-28b2-4460-ade3-21364f6c1c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:39.506568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:39.564472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 736/1474] 10.222.0.49 () {58 vars in 1063 bytes} [Sun Jul 31 18:31:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:39.572338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b62cec-ee4c-4381-8b96-5c008d885b13 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:39.573890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1b62cec-ee4c-4381-8b96-5c008d885b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:39.689199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b62cec-ee4c-4381-8b96-5c008d885b13 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:39.767653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:39.768286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:39.768564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:39.768864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:39.770519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:39.826843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:39.892893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:39.894326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9638ff-2347-4938-bbf6-32343b8271c3 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:39.914065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 739/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:31:41.990973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c09e2-afc6-4cbd-b861-7f49da15dd78 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:41.991893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c09e2-afc6-4cbd-b861-7f49da15dd78 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:41.991893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c09e2-afc6-4cbd-b861-7f49da15dd78 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:41.994070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 737/1476] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:42.003233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d29409-cb0c-470a-9528-0196b2feb506 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:42.003669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d29409-cb0c-470a-9528-0196b2feb506 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:42.003777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d29409-cb0c-470a-9528-0196b2feb506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:42.096678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-62d29409-cb0c-470a-9528-0196b2feb506 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:42.097936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62d29409-cb0c-470a-9528-0196b2feb506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:42.201548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 740/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:42.224397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c09e2-afc6-4cbd-b861-7f49da15dd78 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:42.226286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a11c09e2-afc6-4cbd-b861-7f49da15dd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:42.355610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c09e2-afc6-4cbd-b861-7f49da15dd78 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:42.406671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:42.458495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:42.459129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:42.459406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:42.459654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:42.460356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:42.462177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:31:42.462177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:31:42.462177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:31:42.463324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966977-93c3-42a6-a588-b254b1fedfc1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:42.483311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 738/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Sun Jul 31 18:31:42 2022] GET /identity/v3/projects => generated 2553 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:31:44.476737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4bfbb-2110-4c06-942f-dd8d9171cc5a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:44.477564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4bfbb-2110-4c06-942f-dd8d9171cc5a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:44.477564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4bfbb-2110-4c06-942f-dd8d9171cc5a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:44.479685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 741/1479] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:44.488241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a18830-f10d-45f5-b8fc-1a906809732d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:44.488683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a18830-f10d-45f5-b8fc-1a906809732d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:44.488855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a18830-f10d-45f5-b8fc-1a906809732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:44.569341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-00a18830-f10d-45f5-b8fc-1a906809732d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:44.570560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-00a18830-f10d-45f5-b8fc-1a906809732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:44.677031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 739/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:44.721753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4bfbb-2110-4c06-942f-dd8d9171cc5a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:44.723757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84a4bfbb-2110-4c06-942f-dd8d9171cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:44.737034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:44.883874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4bfbb-2110-4c06-942f-dd8d9171cc5a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:44.981064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:45.016450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:45.017042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:45.017307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:45.017570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:45.019186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:45.108070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:45.109814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0878248-81f7-4568-b850-91ba2a368b9a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:45.127882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 742/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:47.338654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6a10-aff0-4afe-9f3a-314998740eca None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:47.339253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6a10-aff0-4afe-9f3a-314998740eca None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:47.339253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d6a10-aff0-4afe-9f3a-314998740eca None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:47.340599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 740/1482] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:47.347034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33389cf3-1462-4195-b807-c6cea74de021 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:47.347465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33389cf3-1462-4195-b807-c6cea74de021 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:47.347605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33389cf3-1462-4195-b807-c6cea74de021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:47.425506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-33389cf3-1462-4195-b807-c6cea74de021 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:47.426722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-33389cf3-1462-4195-b807-c6cea74de021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:47.474856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:47.528267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 743/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:47.564793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6a10-aff0-4afe-9f3a-314998740eca None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:47.566382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-976d6a10-aff0-4afe-9f3a-314998740eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:47.591325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:47.676807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d6a10-aff0-4afe-9f3a-314998740eca None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:47.752174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:47.752702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:47.752999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:47.753266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:47.755224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:47.830742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:47.866304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:47.867732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b5399-6506-4d1f-ac5f-a58cbafa6459 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:47.886602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 741/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:49.951523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68ddd7-4299-4d9f-b0aa-3f695c7ffcfa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:49.952328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68ddd7-4299-4d9f-b0aa-3f695c7ffcfa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:49.952328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68ddd7-4299-4d9f-b0aa-3f695c7ffcfa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:49.953805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 744/1485] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:49.959087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926893ef-fd76-44fc-84ef-4e1e1d71f5df None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:49.959527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926893ef-fd76-44fc-84ef-4e1e1d71f5df None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:49.959694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926893ef-fd76-44fc-84ef-4e1e1d71f5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:50.029893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-926893ef-fd76-44fc-84ef-4e1e1d71f5df None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:50.031127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-926893ef-fd76-44fc-84ef-4e1e1d71f5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:50.152922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 742/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:50.192050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68ddd7-4299-4d9f-b0aa-3f695c7ffcfa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:50.194195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d68ddd7-4299-4d9f-b0aa-3f695c7ffcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:50.305254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68ddd7-4299-4d9f-b0aa-3f695c7ffcfa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:50.391316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:50.393235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:50.393856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:50.394159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:50.394407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:50.396418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:50.511895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:50.513303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd263f-94af-4322-8dca-91c2c4849348 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:50.531841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 745/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:31:52.709266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae45b6-4011-4e15-aab8-dfaaf8d72cff None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:52.709906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae45b6-4011-4e15-aab8-dfaaf8d72cff None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:52.709906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae45b6-4011-4e15-aab8-dfaaf8d72cff None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:52.711183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 743/1488] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:52.716524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ccf14-c6af-486e-8891-8e60c9dea790 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:52.717017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ccf14-c6af-486e-8891-8e60c9dea790 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:52.717198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ccf14-c6af-486e-8891-8e60c9dea790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:52.788578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f71ccf14-c6af-486e-8891-8e60c9dea790 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:52.789816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f71ccf14-c6af-486e-8891-8e60c9dea790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:52.843888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:52.891029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 746/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:52.937195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae45b6-4011-4e15-aab8-dfaaf8d72cff None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:52.939117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-85ae45b6-4011-4e15-aab8-dfaaf8d72cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:52.982222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:53.056377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae45b6-4011-4e15-aab8-dfaaf8d72cff None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:53.135540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:53.136160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:53.136431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:53.136698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:53.138355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:53.226083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:53.270220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:53.271564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd61e7-5e48-402c-bc9e-909880f006e9 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:53.291293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 744/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:31:55.481882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaab1f-b090-4c62-b5cf-00f643c2d9ce None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:55.482731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaab1f-b090-4c62-b5cf-00f643c2d9ce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:55.482731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaab1f-b090-4c62-b5cf-00f643c2d9ce None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:55.484355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 747/1491] 10.222.0.49 () {54 vars in 926 bytes} [Sun Jul 31 18:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:31:55.489885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cc581-4b7d-4720-82dd-1bb2781ab359 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:55.490311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cc581-4b7d-4720-82dd-1bb2781ab359 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:55.490446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cc581-4b7d-4720-82dd-1bb2781ab359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:55.560729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4f8cc581-4b7d-4720-82dd-1bb2781ab359 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:31:55.562261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4f8cc581-4b7d-4720-82dd-1bb2781ab359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:55.661955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 745/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Sun Jul 31 18:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:31:55.712111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaab1f-b090-4c62-b5cf-00f643c2d9ce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:31:55.713683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5eaab1f-b090-4c62-b5cf-00f643c2d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:55.826013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaab1f-b090-4c62-b5cf-00f643c2d9ce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:31:55.908952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:31:55.909782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:31:55.910057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:31:55.910326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:31:55.911969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:31:55.922132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:31:56.037401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:31:56.038848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853e54b-c1aa-47c6-9b95-ebcc152acb12 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:31:56.062413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 748/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:32:05.660098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=82 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:32:06.058744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=50 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:29.139250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:29.139954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:29.139954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:29.224351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:33:29.225590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:29.335133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 746/1494] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:33:29.344290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:29.344843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:29.344938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:29.425603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:33:29.426713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:29.526019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 749/1495] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:33:29.531239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:29.532528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:29.544569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dde57-d553-4181-bbe2-87a3ed0cb690 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:29.608524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:29.645951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:29.646572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:29.646842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:29.647136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:29.647829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:33:29.648133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:29.649179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:29.649179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:33:29.649179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:33:29.650068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:29.662122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 747/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:33:29.670969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:29.680390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ed812-cc43-4013-8ba6-02120514d31b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:29.745738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:29.781855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:29.782475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:29.782744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:29.783009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:29.783663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:29.785440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:29.785440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:33:29.785440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:33:29.786592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:29.845271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 750/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:33:29 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:33:29.849680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:29.851264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:29.887176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b2a3d-1329-4426-bb33-a113f50bb839 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:29.989675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:29.990215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:29.990487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:29.990756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:29.991428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:29.992441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:29.992441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:33:29.992441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:33:29.993696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:30.039881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 748/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:33:29 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:33:30.044380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:30.045947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:30.110392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934e091-8de3-4738-91dd-0043b48ff196 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:30.203971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:30.215791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:30.216430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:30.216711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:30.217079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:30.217830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:30.218860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:30.218860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:33:30.218860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:33:30.219729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:30.232964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 751/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:33:30.237245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:30.238762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:30.250481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672027c-4a58-43ea-bc56-2de988dc333a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:30.276304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:30.352498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:30.353146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:30.353423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:30.353688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:30.354378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:30.355391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:30.355391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:33:30.355391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:33:30.356278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f133b-014d-4cdb-b6dc-b3201d269a30 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:30.368826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 749/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:33:30.373470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:30.374990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:30.409749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598dfa2-bd77-4cae-bfc0-44b591ea0504 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:30.506563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:30.507143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:30.507419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:30.507689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] PATH_INFO: `/v3/projects/66bd025ac192497f877d23152205f9bd/users/225be69d405c46d880c79bae80a906cf/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:30.529935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bd025ac192497f877d23152205f9bd, user_id=225be69d405c46d880c79bae80a906cf, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:30.530946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:30.530946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:33:30.530946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:33:30.533056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4eedc-8e4d-463f-811e-5abd65c17806 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:30.560047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 752/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:33:30 2022] PUT /identity/v3/projects/66bd025ac192497f877d23152205f9bd/users/225be69d405c46d880c79bae80a906cf/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:33:30.572504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7929577-8830-462f-8419-0d9bc191489e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:30.572785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7929577-8830-462f-8419-0d9bc191489e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:30.573084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7929577-8830-462f-8419-0d9bc191489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:30.636722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b7929577-8830-462f-8419-0d9bc191489e None None] MFA Rules not processed for user `225be69d405c46d880c79bae80a906cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:33:30.638013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b7929577-8830-462f-8419-0d9bc191489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:30.722746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 750/1502] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:33:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:33:30.738326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525921e-d502-4f95-bff0-def0c89d8d59 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:30.738607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525921e-d502-4f95-bff0-def0c89d8d59 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:30.738881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525921e-d502-4f95-bff0-def0c89d8d59 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:30.740281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 753/1503] 10.222.0.49 () {54 vars in 964 bytes} [Sun Jul 31 18:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:33:30.745168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcc2ca-e1db-4bef-8ce0-5abd8be0cc62 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:30.745445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcc2ca-e1db-4bef-8ce0-5abd8be0cc62 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:30.745710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcc2ca-e1db-4bef-8ce0-5abd8be0cc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:30.779066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:30.825636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-15bcc2ca-e1db-4bef-8ce0-5abd8be0cc62 None None] MFA Rules not processed for user `20341bc6269f477c952c25437ab7a2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:33:30.826865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-15bcc2ca-e1db-4bef-8ce0-5abd8be0cc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:30.917395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 751/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Sun Jul 31 18:33:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:33:30.925393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525921e-d502-4f95-bff0-def0c89d8d59 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:30.926935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c525921e-d502-4f95-bff0-def0c89d8d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:30.981797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:31.038460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525921e-d502-4f95-bff0-def0c89d8d59 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:31.116892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:31.117522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:31.117797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:31.118064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:31.119694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:31.226207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:31.228418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:31.233654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:31.250609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 754/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:33:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:33:31.296418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090283b-ddb4-4672-90b7-98b698f83e5b None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:31.296863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090283b-ddb4-4672-90b7-98b698f83e5b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:31.297001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090283b-ddb4-4672-90b7-98b698f83e5b None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:31.298287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 752/1506] 10.222.0.49 () {52 vars in 830 bytes} [Sun Jul 31 18:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:33:31.302167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:31.303399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:31.406570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dac6b-1581-44f2-9b9c-f4779ea0fb46 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:31.502327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:31.502963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:31.503230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:31.503494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] PATH_INFO: `/v3/users/225be69d405c46d880c79bae80a906cf` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:31.516177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] RBAC: Authorizing `identity:delete_user(user_id=225be69d405c46d880c79bae80a906cf)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:31.517234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:31.517234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:33:31.517234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:33:31.518445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:31.528147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:31.598719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '225be69d405c46d880c79bae80a906cf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:31.610634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '225be69d405c46d880c79bae80a906cf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:31.611186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '225be69d405c46d880c79bae80a906cf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:31.636795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '225be69d405c46d880c79bae80a906cf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:31.650954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '225be69d405c46d880c79bae80a906cf'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:31.657098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 755/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:33:31 2022] DELETE /identity/v3/users/225be69d405c46d880c79bae80a906cf => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:33:31.662859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:31.663140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:31.663407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:31.720455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:33:31.721754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:31.800821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:31.815492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 753/1508] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:33:31.826468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:31.827689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:31.936252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb08ae-f910-48b3-b871-92763916152a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:31.991174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:32.012615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:32.013272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:32.013549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:32.013835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:32.015523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:32.131035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:32.132481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a2a41-6477-4408-8505-bcff645f5aef service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:32.151892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 756/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:33:32.497710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:33:32.499018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:33:32.511214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9060106-7c9f-4df0-8ed5-524ab5a211c4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:33:32.609406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:33:32.610285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:33:32.610564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:33:32.610811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] PATH_INFO: `/v3/projects/66bd025ac192497f877d23152205f9bd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:33:32.621608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bd025ac192497f877d23152205f9bd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:33:32.622633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:33:32.622633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:33:32.622633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:33:32.623881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:33:32.685680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bd025ac192497f877d23152205f9bd', 'request_id': 'req-6ea316ba-1ff6-4f40-a807-37c1ca518f17'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:32.698923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bd025ac192497f877d23152205f9bd', 'request_id': 'req-6ea316ba-1ff6-4f40-a807-37c1ca518f17'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:32.705489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bd025ac192497f877d23152205f9bd', 'request_id': 'req-6ea316ba-1ff6-4f40-a807-37c1ca518f17'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:32.731299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] The token cache is being invalidate because project 66bd025ac192497f877d23152205f9bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:33:32.731511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:32.731939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6ea316ba-1ff6-4f40-a807-37c1ca518f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:33:32.733546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 754/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:33:32 2022] DELETE /identity/v3/projects/66bd025ac192497f877d23152205f9bd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:33:42.149546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=62 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:33:42.729253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:35:10.505254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b2978-f5d3-4381-8fe8-2907a72ef807 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:10.505894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b2978-f5d3-4381-8fe8-2907a72ef807 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:10.505894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b2978-f5d3-4381-8fe8-2907a72ef807 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:10.507606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:35:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:10.512845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c36-381c-4cf5-94f5-da2626f9e0bf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:10.513288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c36-381c-4cf5-94f5-da2626f9e0bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:10.513389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c36-381c-4cf5-94f5-da2626f9e0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:10.603018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-eacb7c36-381c-4cf5-94f5-da2626f9e0bf None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:10.604237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eacb7c36-381c-4cf5-94f5-da2626f9e0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:10.716782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:35:10.874093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b2978-f5d3-4381-8fe8-2907a72ef807 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:10.877906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-678b2978-f5d3-4381-8fe8-2907a72ef807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:11.005698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b2978-f5d3-4381-8fe8-2907a72ef807 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:11.012115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a1c5a-081b-45f9-b8fc-e0166fde2723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:11.012739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a1c5a-081b-45f9-b8fc-e0166fde2723 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:11.013048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a1c5a-081b-45f9-b8fc-e0166fde2723 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:11.013300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a1c5a-081b-45f9-b8fc-e0166fde2723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:11.022092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a1c5a-081b-45f9-b8fc-e0166fde2723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:11.023678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a1c5a-081b-45f9-b8fc-e0166fde2723 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:11.034386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 758/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:11.874088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2c065-f078-4c13-b554-56a20be9f11f None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:11.875024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2c065-f078-4c13-b554-56a20be9f11f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:11.875024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2c065-f078-4c13-b554-56a20be9f11f None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:11.877126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:35:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:11.882642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b363-f0bb-4f22-9e97-2dba5cbbbd9e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:11.883087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b363-f0bb-4f22-9e97-2dba5cbbbd9e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:11.883199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b363-f0bb-4f22-9e97-2dba5cbbbd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:11.904471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c709b363-f0bb-4f22-9e97-2dba5cbbbd9e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:11.905621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c709b363-f0bb-4f22-9e97-2dba5cbbbd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:11.917675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:12.069780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2c065-f078-4c13-b554-56a20be9f11f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:12.090451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2c065-f078-4c13-b554-56a20be9f11f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:12.096957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f6239-b284-44dd-b862-fd86e4903494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:12.097564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f6239-b284-44dd-b862-fd86e4903494 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:12.097834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f6239-b284-44dd-b862-fd86e4903494 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:12.098098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f6239-b284-44dd-b862-fd86e4903494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:12.106599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f6239-b284-44dd-b862-fd86e4903494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:12.108186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f6239-b284-44dd-b862-fd86e4903494 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:12.118286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 757/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:12.836244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9b4d-9bee-48d8-8684-b1ea532ce4f5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:12.836482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9b4d-9bee-48d8-8684-b1ea532ce4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:12.836698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9b4d-9bee-48d8-8684-b1ea532ce4f5 None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:12.838070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 760/1517] 10.222.0.57 () {54 vars in 925 bytes} [Sun Jul 31 18:35:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:35:12.844993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeae3c-216b-4100-ae14-acb5eb704de0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:12.845275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeae3c-216b-4100-ae14-acb5eb704de0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:12.845553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeae3c-216b-4100-ae14-acb5eb704de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:12.907224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d0eeae3c-216b-4100-ae14-acb5eb704de0 None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:12.908421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d0eeae3c-216b-4100-ae14-acb5eb704de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:12.926184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:35:12.983568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 758/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Sun Jul 31 18:35:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:16.850715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482287e5-b739-4f8f-8af0-27802ca9f005 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:16.851590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482287e5-b739-4f8f-8af0-27802ca9f005 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:16.851590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482287e5-b739-4f8f-8af0-27802ca9f005 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:16.853327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 761/1519] 10.222.0.49 () {54 vars in 961 bytes} [Sun Jul 31 18:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:16.859315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57c919-88fd-43ae-a1d0-da5da0f05785 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:16.859839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57c919-88fd-43ae-a1d0-da5da0f05785 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:16.859950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57c919-88fd-43ae-a1d0-da5da0f05785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:16.888517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b57c919-88fd-43ae-a1d0-da5da0f05785 None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:16.889746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b57c919-88fd-43ae-a1d0-da5da0f05785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:16.900296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 759/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Sun Jul 31 18:35:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:16.911025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482287e5-b739-4f8f-8af0-27802ca9f005 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:16.930440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482287e5-b739-4f8f-8af0-27802ca9f005 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:16.935857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f5364-b9f2-4e25-af84-91a95f62ddf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:16.936464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5364-b9f2-4e25-af84-91a95f62ddf4 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:16.936733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5364-b9f2-4e25-af84-91a95f62ddf4 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:16.937035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f5364-b9f2-4e25-af84-91a95f62ddf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:16.945520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f5364-b9f2-4e25-af84-91a95f62ddf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:16.947104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f5364-b9f2-4e25-af84-91a95f62ddf4 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:16.956595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 762/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:23.352572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7def7-fd72-4658-ab0a-82dce3769794 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:23.353199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7def7-fd72-4658-ab0a-82dce3769794 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:23.353199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7def7-fd72-4658-ab0a-82dce3769794 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:23.354489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:35:23.360411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece799ac-82c1-45b3-b550-4ea094b6f72d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:23.361232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece799ac-82c1-45b3-b550-4ea094b6f72d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:23.361404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece799ac-82c1-45b3-b550-4ea094b6f72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:23.384482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ece799ac-82c1-45b3-b550-4ea094b6f72d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:23.385714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ece799ac-82c1-45b3-b550-4ea094b6f72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:23.395429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:23.550454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7def7-fd72-4658-ab0a-82dce3769794 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:23.572072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7def7-fd72-4658-ab0a-82dce3769794 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:23.578091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9fe5c-c16f-4d96-a69a-9aabbd15ea3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:23.578679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9fe5c-c16f-4d96-a69a-9aabbd15ea3b service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:23.578949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9fe5c-c16f-4d96-a69a-9aabbd15ea3b service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:23.579215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9fe5c-c16f-4d96-a69a-9aabbd15ea3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:23.587661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9fe5c-c16f-4d96-a69a-9aabbd15ea3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:23.589392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9fe5c-c16f-4d96-a69a-9aabbd15ea3b service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:23.599193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 761/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:24.418770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21261c6-45bb-4ed2-8249-10fdd974b3fe None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:24.419417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21261c6-45bb-4ed2-8249-10fdd974b3fe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:24.419417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21261c6-45bb-4ed2-8249-10fdd974b3fe None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:24.420719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 18:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:24.428830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadeb27-ed1a-4e3c-8634-616bccfcabbf None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:24.429590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadeb27-ed1a-4e3c-8634-616bccfcabbf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:24.429755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadeb27-ed1a-4e3c-8634-616bccfcabbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:24.455894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cdadeb27-ed1a-4e3c-8634-616bccfcabbf None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:24.457137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cdadeb27-ed1a-4e3c-8634-616bccfcabbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:24.466473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 18:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:24.619026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21261c6-45bb-4ed2-8249-10fdd974b3fe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:24.645644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21261c6-45bb-4ed2-8249-10fdd974b3fe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:24.652993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69b8e-26a6-47f3-950a-79d716ca4b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:24.653606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69b8e-26a6-47f3-950a-79d716ca4b13 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:24.653882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69b8e-26a6-47f3-950a-79d716ca4b13 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:24.654149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69b8e-26a6-47f3-950a-79d716ca4b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:24.662687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d69b8e-26a6-47f3-950a-79d716ca4b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:24.664273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d69b8e-26a6-47f3-950a-79d716ca4b13 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:24.673809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 765/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:28.430358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1827d09-e3b2-41d9-b59e-9cd8967a64d2 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:28.430987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1827d09-e3b2-41d9-b59e-9cd8967a64d2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:28.430987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1827d09-e3b2-41d9-b59e-9cd8967a64d2 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:28.432254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:28.438722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5cc89-7c2d-4fff-a55e-d56667e3ba7e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:28.439514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5cc89-7c2d-4fff-a55e-d56667e3ba7e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:28.439514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5cc89-7c2d-4fff-a55e-d56667e3ba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:28.468260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e7d5cc89-7c2d-4fff-a55e-d56667e3ba7e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:28.469499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e7d5cc89-7c2d-4fff-a55e-d56667e3ba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:28.478538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:28.555894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1827d09-e3b2-41d9-b59e-9cd8967a64d2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:28.557713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f1827d09-e3b2-41d9-b59e-9cd8967a64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:28.638724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1827d09-e3b2-41d9-b59e-9cd8967a64d2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:28.642873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aacf7-27b8-48f7-be65-f24198ec3c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:28.643486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aacf7-27b8-48f7-be65-f24198ec3c60 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:28.643753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aacf7-27b8-48f7-be65-f24198ec3c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:28.644024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aacf7-27b8-48f7-be65-f24198ec3c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:28.653649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aacf7-27b8-48f7-be65-f24198ec3c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:28.655064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aacf7-27b8-48f7-be65-f24198ec3c60 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:28.665215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 764/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:30.635145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06aa29b-ebed-4fa2-aa27-0631a3276a5a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:30.635524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06aa29b-ebed-4fa2-aa27-0631a3276a5a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:30.635621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06aa29b-ebed-4fa2-aa27-0631a3276a5a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:30.637759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:30.646173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea413369-36d9-47b0-8e31-59b0c773fc08 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:30.646695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea413369-36d9-47b0-8e31-59b0c773fc08 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:30.646798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea413369-36d9-47b0-8e31-59b0c773fc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:30.666934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ea413369-36d9-47b0-8e31-59b0c773fc08 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:30.668127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ea413369-36d9-47b0-8e31-59b0c773fc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:30.676633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:30.712689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06aa29b-ebed-4fa2-aa27-0631a3276a5a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:30.714841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c06aa29b-ebed-4fa2-aa27-0631a3276a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:30.729314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06aa29b-ebed-4fa2-aa27-0631a3276a5a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:30.733218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5aef5-4f93-45ea-909b-aafc4c5d1e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:30.733833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5aef5-4f93-45ea-909b-aafc4c5d1e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:30.734109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5aef5-4f93-45ea-909b-aafc4c5d1e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:30.734378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5aef5-4f93-45ea-909b-aafc4c5d1e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:30.743608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5aef5-4f93-45ea-909b-aafc4c5d1e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:30.745072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5aef5-4f93-45ea-909b-aafc4c5d1e84 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:30.754916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 768/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:34.353329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e61c3-03d2-4d8b-96da-7f730048edf9 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:34.353917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e61c3-03d2-4d8b-96da-7f730048edf9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:34.353917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e61c3-03d2-4d8b-96da-7f730048edf9 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:34.355222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:34.360674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197fa1-ea24-4108-aff8-607c13384ee2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:34.361185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197fa1-ea24-4108-aff8-607c13384ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:34.361295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197fa1-ea24-4108-aff8-607c13384ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:34.371328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:35:34.388786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9b197fa1-ea24-4108-aff8-607c13384ee2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:34.392436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9b197fa1-ea24-4108-aff8-607c13384ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:34.401370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:34.734092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e61c3-03d2-4d8b-96da-7f730048edf9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:34.737368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b95e61c3-03d2-4d8b-96da-7f730048edf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:34.820833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:35:34.840808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e61c3-03d2-4d8b-96da-7f730048edf9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:34.846850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef9282-4aa0-4810-b965-7772696e603a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:34.847701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9282-4aa0-4810-b965-7772696e603a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:34.847935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9282-4aa0-4810-b965-7772696e603a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:34.848303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9282-4aa0-4810-b965-7772696e603a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:34.858373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef9282-4aa0-4810-b965-7772696e603a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:34.859808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef9282-4aa0-4810-b965-7772696e603a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:34.869577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 767/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:36.872427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa944e-8725-480d-b55c-68b334a784c5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:36.872994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa944e-8725-480d-b55c-68b334a784c5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:36.872994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa944e-8725-480d-b55c-68b334a784c5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:36.874278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 18:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:35:36.881111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4370d-908a-4b69-9f4f-02893dd0f5d7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:36.881844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4370d-908a-4b69-9f4f-02893dd0f5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:36.881995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4370d-908a-4b69-9f4f-02893dd0f5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:36.908087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f0d4370d-908a-4b69-9f4f-02893dd0f5d7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:35:36.909330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f0d4370d-908a-4b69-9f4f-02893dd0f5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:36.918840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 18:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:35:37.223498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa944e-8725-480d-b55c-68b334a784c5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:35:37.225494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bcfa944e-8725-480d-b55c-68b334a784c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:35:37.239473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa944e-8725-480d-b55c-68b334a784c5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:35:37.243950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2480b0-1e7a-41bc-a50e-f0574e6779db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:35:37.244561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2480b0-1e7a-41bc-a50e-f0574e6779db service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:35:37.244841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2480b0-1e7a-41bc-a50e-f0574e6779db service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:35:37.245124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2480b0-1e7a-41bc-a50e-f0574e6779db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:35:37.254963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2480b0-1e7a-41bc-a50e-f0574e6779db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:35:37.256412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2480b0-1e7a-41bc-a50e-f0574e6779db service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:35:37.265717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 771/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:35:46.903230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=11 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:35:47.261723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:05.452194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.452955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.452955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.478547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.479745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.491845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 769/1540] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:05.501347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821d77c-41bb-41be-be61-6bf49abe70c1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.501920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821d77c-41bb-41be-be61-6bf49abe70c1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.501920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821d77c-41bb-41be-be61-6bf49abe70c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.530467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8821d77c-41bb-41be-be61-6bf49abe70c1 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.531685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8821d77c-41bb-41be-be61-6bf49abe70c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.546676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 772/1541] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.551179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:05.567318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487d9c-ab9c-49ea-8121-ae0c831abe90 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:05.571042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723a0d1-30f3-4f12-a073-adf54282fe97 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.571330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723a0d1-30f3-4f12-a073-adf54282fe97 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.571757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723a0d1-30f3-4f12-a073-adf54282fe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.573583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:05.574207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.574470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.574743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.575491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:05.575797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:05.577556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:05.602132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8723a0d1-30f3-4f12-a073-adf54282fe97 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.602485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 770/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:05.603376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8723a0d1-30f3-4f12-a073-adf54282fe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.612728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:05.619539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 773/1543] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.631265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38541f63-3e12-43d2-8a18-b3be0743a1c8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:05.640191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:05.640848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.641121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.641386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.642055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:05.643077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:37:05.643077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:37:05.643077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:37:05.644274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:05.652337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b2bf-58cb-4f69-be4c-20981b6814ab None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.652629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b2bf-58cb-4f69-be4c-20981b6814ab None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.652933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8b2bf-58cb-4f69-be4c-20981b6814ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.689839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b6a8b2bf-58cb-4f69-be4c-20981b6814ab None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.691060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6a8b2bf-58cb-4f69-be4c-20981b6814ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.710698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 774/1544] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:05.711975 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.712314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.712691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.721386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 771/1545] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:05.727059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:05.749697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.752092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.808220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641444bc-4d44-443b-abde-9078bbf918ac admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:05.813522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4049b-d197-4a6c-bd0d-155e40ad4dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:05.814147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4049b-d197-4a6c-bd0d-155e40ad4dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.814553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4049b-d197-4a6c-bd0d-155e40ad4dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.814823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4049b-d197-4a6c-bd0d-155e40ad4dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.815507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4049b-d197-4a6c-bd0d-155e40ad4dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:05.817719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4049b-d197-4a6c-bd0d-155e40ad4dcc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:05.821929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 775/1546] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.823246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:05.834310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d00f6-e83e-4ac9-b9cd-0604ffb55bb6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:05.839671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:05.840293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.840561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.840877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.841550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:05.841841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:05.843011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:37:05.843011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:37:05.843011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:37:05.843486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 772/1547] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:05.843975 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00e590-c7b9-4ea9-9b49-718e278d8ebc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:05.844747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0599b2-2023-4286-89ec-06735dc64b09 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.848107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0599b2-2023-4286-89ec-06735dc64b09 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.848390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0599b2-2023-4286-89ec-06735dc64b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.857518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 776/1548] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:05.858753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e5b02-0195-4045-90b3-e4d657967b2d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.859034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e5b02-0195-4045-90b3-e4d657967b2d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.859298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e5b02-0195-4045-90b3-e4d657967b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.875016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ae0599b2-2023-4286-89ec-06735dc64b09 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.876219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ae0599b2-2023-4286-89ec-06735dc64b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.882367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-384e5b02-0195-4045-90b3-e4d657967b2d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.883506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-384e5b02-0195-4045-90b3-e4d657967b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.884590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 773/1549] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.885793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e741e9b-667b-4e44-a0fd-119c71d0f223 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.886065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e741e9b-667b-4e44-a0fd-119c71d0f223 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.886329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e741e9b-667b-4e44-a0fd-119c71d0f223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.896981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 777/1550] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.898666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.899014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.899290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.905420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e741e9b-667b-4e44-a0fd-119c71d0f223 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.906602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e741e9b-667b-4e44-a0fd-119c71d0f223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.914791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 774/1551] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.916141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.916426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.916699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.919012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.920255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.928615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 778/1552] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.929258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:05.936878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.938088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.939221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344c572-10f4-4239-b031-b19cdbfc8f7f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:05.943744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92894bd8-3aaf-4f18-8fa2-1461c73daa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:05.944330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92894bd8-3aaf-4f18-8fa2-1461c73daa0e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.944601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92894bd8-3aaf-4f18-8fa2-1461c73daa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.944896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92894bd8-3aaf-4f18-8fa2-1461c73daa0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.945596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92894bd8-3aaf-4f18-8fa2-1461c73daa0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:05.949681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 775/1553] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.949835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92894bd8-3aaf-4f18-8fa2-1461c73daa0e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:05.949950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:05.958719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44658e-6670-424b-a078-0ab998a32e6c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:05.959797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 779/1554] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:05.961200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8598-6dd6-4363-94bf-bbe6ac35caf8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.961448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8598-6dd6-4363-94bf-bbe6ac35caf8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.961721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8598-6dd6-4363-94bf-bbe6ac35caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.963438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dabba-4e26-439b-903a-fb8cb55f28a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:05.964045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dabba-4e26-439b-903a-fb8cb55f28a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.964322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dabba-4e26-439b-903a-fb8cb55f28a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.964592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dabba-4e26-439b-903a-fb8cb55f28a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:05.965280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dabba-4e26-439b-903a-fb8cb55f28a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:05.967298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dabba-4e26-439b-903a-fb8cb55f28a8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:05.981992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6acc8598-6dd6-4363-94bf-bbe6ac35caf8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:05.983319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6acc8598-6dd6-4363-94bf-bbe6ac35caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:05.992819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 780/1555] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:05.994854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd2204-a0b4-4d2b-8c86-3efcc4246f23 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:05.995260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd2204-a0b4-4d2b-8c86-3efcc4246f23 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:05.995682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd2204-a0b4-4d2b-8c86-3efcc4246f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.002366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 776/1556] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.003625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f24fb-82cd-44d9-8988-bda7c5e444f3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.003906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f24fb-82cd-44d9-8988-bda7c5e444f3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.004660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f24fb-82cd-44d9-8988-bda7c5e444f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.018563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d1cd2204-a0b4-4d2b-8c86-3efcc4246f23 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:06.020060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d1cd2204-a0b4-4d2b-8c86-3efcc4246f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:06.024500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e0f24fb-82cd-44d9-8988-bda7c5e444f3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:06.025704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e0f24fb-82cd-44d9-8988-bda7c5e444f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:06.075008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:06.077126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 781/1557] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:06.078279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.078560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.078828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.087973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 777/1558] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:06.089261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.089545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.089817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.100559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:06.101781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:06.108471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:06.109711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:06.110120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 782/1559] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:06.110951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.120143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e947b9-b2e1-40c6-b4da-27951c4a4a1a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.121402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 778/1560] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:06.124449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.125309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.125942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.126215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.126479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.127195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:06.127471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.129452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf05eca-1923-4132-bfea-035e1b0e32ff admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.134126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fae7f-d888-44e6-86c5-590a31beae8d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.138806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.139431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.139713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.139981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.140664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.141063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 783/1561] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:06.148385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.148512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.148608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.148608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.152723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 779/1562] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:06.153592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.163018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e830-e6d2-4fc9-8bda-c46f235750ca admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.163227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:06.164409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:06.167556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.168139 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.168413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.168828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.169479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.171443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.172377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 784/1563] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:06.173167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.182525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435e653-cbca-4a4b-a5b5-e463d6aab1ea None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.187257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.187787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.188064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.188339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.189045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:06.189339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.190931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.196668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 780/1564] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 288 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.196668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.197956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:06.203479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 785/1565] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:06.204597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fa358-9133-4cb4-8d73-b2bdfbd074f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.205079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.209320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.209907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.210178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.210438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.211122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:06.211490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.213025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.219126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1112a-e0a5-4943-916a-f4f6f80c5ec5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.224407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.225055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.225383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.225651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.226343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:06.226635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.227129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 781/1566] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:06.227969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.228406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.239526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b345935-e71d-4332-a39b-b5676d0456b4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.247984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 786/1567] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:06.247984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.249788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.250443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.250717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.251034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.251998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:06.252298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.254318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.255825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc268-545d-475e-a3e3-965854839ea3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.260782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.261544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.261815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.262078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.262734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.264642 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.266204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 782/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:06.266964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.276517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cc0a8-50b2-4059-8c16-f6f0e6336abf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.281081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.281727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.282004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.282272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] PATH_INFO: `/v3/projects/a2eb8b8a120e4e87b4407d0abb3b3f07/users/e518e5691a7247de9d04ae9828809498/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.292568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 787/1569] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.293487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.297355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2eb8b8a120e4e87b4407d0abb3b3f07, user_id=e518e5691a7247de9d04ae9828809498, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.298379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:37:06.298379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:37:06.298379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:37:06.300306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.314918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 783/1570] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:06 2022] PUT /identity/v3/projects/a2eb8b8a120e4e87b4407d0abb3b3f07/users/e518e5691a7247de9d04ae9828809498/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:06.316049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.363191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661260c-ad9e-415c-b72f-e5b434a7c767 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.379875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce0c7c-d1fe-418b-b2c1-4cc6fce79010 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.385387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.386004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.386273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.386562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.387259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.388861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.398679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 784/1571] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:06.399437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.408871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0104ed-ae11-4236-9284-641c79142896 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.413705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.414472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.414956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.415230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.415889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.417793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.419424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.420056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.420335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.420605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.421331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:06.421632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.423226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.434990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 788/1572] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:06.435988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.447503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981eeee-d8c1-4888-b4ec-e6313146c287 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.447839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 785/1573] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.448684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.494977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:06.502684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.503317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.503593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.503860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.504922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.506843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.509807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fc386-1255-42a4-b199-6d827b5c806d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.514382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.514996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.515270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.515538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.516282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.516862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:06.518186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.540752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 789/1574] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.541885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.557429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 786/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.558240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.610307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a425a2-fc8e-4da8-b4c2-82c0fc451467 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.615094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05709c87-3e47-4924-b88f-d033b7a75664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.615720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05709c87-3e47-4924-b88f-d033b7a75664 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.616018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05709c87-3e47-4924-b88f-d033b7a75664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.616274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05709c87-3e47-4924-b88f-d033b7a75664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.616974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05709c87-3e47-4924-b88f-d033b7a75664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.618926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05709c87-3e47-4924-b88f-d033b7a75664 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.619249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed46cf9-e21c-42f8-926d-5a0540f7b207 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.624078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.624687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.625016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.625278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.625930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.627825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.647372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 790/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.648495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.648800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.649081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.653717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 787/1577] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.654766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.666179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425c0a-8b35-41cc-bf02-86398c4b3f47 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.668611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:06.669846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:06.725062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.725688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.725968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.726229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.726947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.727154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 791/1578] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:06.727932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.728739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.737342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b315-6825-4c36-90cb-2ccaa4845a13 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.740582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 788/1579] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:06.741489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.742431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.743053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.743350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.743744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.744251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.746250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.751106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3f104-fa1d-41fe-84bb-4a110363a32c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.756130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.756715 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.757136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.757402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.758071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.759963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.774265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 792/1580] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.775063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.785537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 789/1581] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.788862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.796000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd862f1-df57-4d98-a555-d3acde9b95f3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.830193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:06.836213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:06.843782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5122-71db-499a-9f80-2067af700cde admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.848327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.849047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.849322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.849584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.850241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.851260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:37:06.851260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:37:06.851260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:37:06.852468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.857388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.858015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.858369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.858787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.859531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.861564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.874240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 793/1582] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.876817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.884931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 790/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.885866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.887350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292632f9-f5fd-40dd-b24d-bd9a87035c1c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.892316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.892959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.893235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.893495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.894151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.895352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f6b91-afeb-40f2-8d14-5ffe71494c4d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.896077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.899848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.900446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.900753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.901025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.901706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.903266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.917867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 791/1584] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:06.918875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.924924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 794/1585] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:06.924924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.933607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec74a4-1801-464c-8b91-8461734a3072 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:06.938681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:06.939353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:06.939749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:06.940028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] PATH_INFO: `/v3/projects/df41262534584aa1930c9960bde9d2a8/users/6e196cc129c342628a657636e0a3610f/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:06.956123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df41262534584aa1930c9960bde9d2a8, user_id=6e196cc129c342628a657636e0a3610f, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:06.958832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:06.975331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 795/1586] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:06 2022] PUT /identity/v3/projects/df41262534584aa1930c9960bde9d2a8/users/6e196cc129c342628a657636e0a3610f/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:06.976416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:06.982699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ba9d7-31e0-41e6-892c-41f13994c9e0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.033502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.034030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.034308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.034584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.041647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310f9b1-bb5c-457b-9a11-975ab990619c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.043628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.045089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.046220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.046840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.047111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.047373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.048026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.049957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.054454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 792/1587] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:07.055140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.064296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393096f-9543-4f4b-9400-7844c3f5a4b1 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.070675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.071426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.071765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.072099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.072893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.077036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 796/1588] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:06 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:07.077036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.077300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.086952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869cecf-d520-4058-9900-0d8f51130f6d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.087522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 793/1589] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.088370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.094764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.095400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.095684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.095952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.096650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.098299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.098785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d2425-56dc-4110-a16e-69f4e398e560 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.103660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.104296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.104574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.104884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.105578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.107157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.112684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 797/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.112684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.121326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 794/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.121839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0285b9-2c75-4e02-93ad-aa45d9601592 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.125753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.129888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae708d-4c4f-4b04-92ac-b9dc6f3ee36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.130542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae708d-4c4f-4b04-92ac-b9dc6f3ee36f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.130796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae708d-4c4f-4b04-92ac-b9dc6f3ee36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.131064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae708d-4c4f-4b04-92ac-b9dc6f3ee36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.131786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae708d-4c4f-4b04-92ac-b9dc6f3ee36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.134327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae708d-4c4f-4b04-92ac-b9dc6f3ee36f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.138096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60762d-aac7-4da7-9da1-4e4b134811de admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.143333 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.144131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.144412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.144681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.145636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.146207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 798/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.147573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.150784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.151011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.151259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.164140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 795/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.164140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.174886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd175f0c-5a8e-45ae-b006-4d7f83dcdc71 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.180213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.180811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.180947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] MFA Rules not processed for user `6e196cc129c342628a657636e0a3610f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:07.181141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.181391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.182063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.182200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:07.183702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.194596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 796/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.198829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.208308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f277e5-28e0-41e1-be23-2c5895e56dab admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.215121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.216625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.217187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.217646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.218548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.221209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.235057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 797/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.236301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.246680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f65f9-7364-4ec1-b1e9-19d19827e0ec admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.251608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.252229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.252504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.252780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.253652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.255548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.263210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:07.269432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 799/1596] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:07.270473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 798/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.270629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.281218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.300227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcb44f-f4cf-4a1b-8540-232dff0ce89a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.301243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557654ab-3665-49cb-95cd-7213494226a7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.308239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.309200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.309486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.309792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.310531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.311489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.312163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.313190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.313426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.313899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] PATH_INFO: `/v3/projects/2269cace554c401a8b6a6303ff4e4da8/users/120c06d6dee6425ea5366b363d5a5f7e/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.325767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:07.326379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 800/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.327280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.336045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2269cace554c401a8b6a6303ff4e4da8, user_id=120c06d6dee6425ea5366b363d5a5f7e, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.339124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27231a-3e79-4bf2-8978-2a6bf2641a1c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.339888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.344447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.345398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.345685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.345948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.346648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.348362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.355754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 799/1599] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:07 2022] PUT /identity/v3/projects/2269cace554c401a8b6a6303ff4e4da8/users/120c06d6dee6425ea5366b363d5a5f7e/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:07.362340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 801/1600] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.362340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.362569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.373349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd920-82a5-46e1-8e43-45541fee6d58 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.442801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f19c02-2f9e-48e4-9b7c-7467a393f0f6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.448729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.449453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.449821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.450199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.450904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.452717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.452905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb66bf-a9ad-408f-b915-f514909d89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.453658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb66bf-a9ad-408f-b915-f514909d89f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.454003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb66bf-a9ad-408f-b915-f514909d89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.454344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb66bf-a9ad-408f-b915-f514909d89f5 admin admin] PATH_INFO: `/v3/projects/c36ffef5602f4ffd806ae72e49d3b339/users/3aaaec3395b04183926746825115ec62/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.468906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 800/1601] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:07.468906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.470836 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb66bf-a9ad-408f-b915-f514909d89f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36ffef5602f4ffd806ae72e49d3b339, user_id=3aaaec3395b04183926746825115ec62, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.473688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb66bf-a9ad-408f-b915-f514909d89f5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.478799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8970450-9f56-4d5b-b8b9-07c59145d05d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.484016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5d547-d3d8-4de5-a6b2-7ba310e0ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.484859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5d547-d3d8-4de5-a6b2-7ba310e0ef30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.485063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5d547-d3d8-4de5-a6b2-7ba310e0ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.485498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5d547-d3d8-4de5-a6b2-7ba310e0ef30 admin admin] PATH_INFO: `/v3/projects/fc4113d291354d99b2591289639ec248/users/4dc88d9b0bc743119dcb7dc2fa8fdcd6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.491551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 802/1602] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:07 2022] PUT /identity/v3/projects/c36ffef5602f4ffd806ae72e49d3b339/users/3aaaec3395b04183926746825115ec62/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:07.494426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036dec7-5846-4e99-ad0d-6d0c3f41f3af None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.494912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036dec7-5846-4e99-ad0d-6d0c3f41f3af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.495527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036dec7-5846-4e99-ad0d-6d0c3f41f3af None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.497402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 803/1603] 10.222.0.49 () {54 vars in 962 bytes} [Sun Jul 31 18:37:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:07.498456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036dec7-5846-4e99-ad0d-6d0c3f41f3af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.513480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5d547-d3d8-4de5-a6b2-7ba310e0ef30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4113d291354d99b2591289639ec248, user_id=4dc88d9b0bc743119dcb7dc2fa8fdcd6, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.516998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5d547-d3d8-4de5-a6b2-7ba310e0ef30 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.534845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 801/1604] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:07 2022] PUT /identity/v3/projects/fc4113d291354d99b2591289639ec248/users/4dc88d9b0bc743119dcb7dc2fa8fdcd6/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:07.536067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.536378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.536995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.558006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:07.560079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:07.588269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036dec7-5846-4e99-ad0d-6d0c3f41f3af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.634517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 802/1605] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:07.636025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.651291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.651973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.652244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.652507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] PATH_INFO: `/v3/projects/8b2df629457c4ff68ad061eb29cb232f/users/88472fa130654657af599106c6aac279/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.653371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d222a-eaa4-46ab-ae00-2eb74f24a300 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.659636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9466-f2ef-45bb-9317-93e35f29a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.660406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9466-f2ef-45bb-9317-93e35f29a90c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.660841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9466-f2ef-45bb-9317-93e35f29a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.661124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9466-f2ef-45bb-9317-93e35f29a90c admin admin] PATH_INFO: `/v3/projects/df42ccb208664e04ac753b0c64b4fab7/users/0d7ed577305d48c795f280e49b438bfc/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.666207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:07.671405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2df629457c4ff68ad061eb29cb232f, user_id=88472fa130654657af599106c6aac279, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.673796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.679264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c9466-f2ef-45bb-9317-93e35f29a90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df42ccb208664e04ac753b0c64b4fab7, user_id=0d7ed577305d48c795f280e49b438bfc, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.682509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c9466-f2ef-45bb-9317-93e35f29a90c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.690521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 804/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:07 2022] PUT /identity/v3/projects/8b2df629457c4ff68ad061eb29cb232f/users/88472fa130654657af599106c6aac279/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:07.691554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:07.702698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 803/1607] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:07 2022] PUT /identity/v3/projects/df42ccb208664e04ac753b0c64b4fab7/users/0d7ed577305d48c795f280e49b438bfc/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:07.704104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56fa38-8037-4168-8516-e7f1d06efa98 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.704448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56fa38-8037-4168-8516-e7f1d06efa98 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.704803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56fa38-8037-4168-8516-e7f1d06efa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.759175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4a56fa38-8037-4168-8516-e7f1d06efa98 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:07.760549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4a56fa38-8037-4168-8516-e7f1d06efa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:07.769842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12217702-51e5-4e45-bad3-994a37934cf4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:07.781998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:07.844197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb55fa-21e5-4e20-9632-191ed69b50a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:07.845378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb55fa-21e5-4e20-9632-191ed69b50a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.845378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb55fa-21e5-4e20-9632-191ed69b50a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.845623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb55fa-21e5-4e20-9632-191ed69b50a8 admin admin] PATH_INFO: `/v3/projects/5737a28d766849f8ab641c4594ed46f0/users/0b3e9aa34d474149a005ee8df63822fd/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.853520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 804/1608] 10.222.0.49 () {58 vars in 1059 bytes} [Sun Jul 31 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:07.855042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.855407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.855730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.863912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb55fa-21e5-4e20-9632-191ed69b50a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5737a28d766849f8ab641c4594ed46f0, user_id=0b3e9aa34d474149a005ee8df63822fd, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:07.866967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb55fa-21e5-4e20-9632-191ed69b50a8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:07.885923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 805/1609] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:07 2022] PUT /identity/v3/projects/5737a28d766849f8ab641c4594ed46f0/users/0b3e9aa34d474149a005ee8df63822fd/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:07.886900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f5f1f-00ef-4726-a4e8-1277f2471da3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:07.887180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f5f1f-00ef-4726-a4e8-1277f2471da3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:07.887450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f5f1f-00ef-4726-a4e8-1277f2471da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:07.906312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] MFA Rules not processed for user `3aaaec3395b04183926746825115ec62`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:07.907633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:07.918466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9d6f5f1f-00ef-4726-a4e8-1277f2471da3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:07.919781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d6f5f1f-00ef-4726-a4e8-1277f2471da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:07.992745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 805/1610] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:07.993867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.020449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 806/1611] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:08.021800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be228e-a1c1-4a22-8f09-5c418f07bdfe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.022077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be228e-a1c1-4a22-8f09-5c418f07bdfe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.024475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be228e-a1c1-4a22-8f09-5c418f07bdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.061013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c8be228e-a1c1-4a22-8f09-5c418f07bdfe None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:08.063423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c8be228e-a1c1-4a22-8f09-5c418f07bdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:08.073501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 807/1612] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:08.078305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.078586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.078856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.090776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40293b-641d-4f74-929d-9bcf5794264f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.096527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84be60-9aaa-4143-b5c9-6c467ec07013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.097377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84be60-9aaa-4143-b5c9-6c467ec07013 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.097752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84be60-9aaa-4143-b5c9-6c467ec07013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.098573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84be60-9aaa-4143-b5c9-6c467ec07013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.103569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:08.105010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:08.111566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84be60-9aaa-4143-b5c9-6c467ec07013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.113279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84be60-9aaa-4143-b5c9-6c467ec07013 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.121050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 808/1613] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:08.122076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.124318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 806/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.125767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d228-44e4-4c28-b248-0d3a8aa1c578 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.126282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d228-44e4-4c28-b248-0d3a8aa1c578 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.126438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d228-44e4-4c28-b248-0d3a8aa1c578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.140407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:08.158949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b239d228-44e4-4c28-b248-0d3a8aa1c578 None None] MFA Rules not processed for user `0b3e9aa34d474149a005ee8df63822fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:08.160446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b239d228-44e4-4c28-b248-0d3a8aa1c578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:08.215508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:08.249312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903a8ba-df8a-4812-870a-cba58f690f27 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.257254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.257984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.258385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.258682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.258800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 807/1615] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:08.260407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d74310-f509-4b5c-bff5-66fd294996bb None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.260810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d74310-f509-4b5c-bff5-66fd294996bb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.261130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d74310-f509-4b5c-bff5-66fd294996bb None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.262810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 808/1616] 10.222.0.49 () {54 vars in 962 bytes} [Sun Jul 31 18:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:08.263937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d74310-f509-4b5c-bff5-66fd294996bb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.277276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d74310-f509-4b5c-bff5-66fd294996bb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.281465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.282168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.282442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.282794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.293246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.294781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.309094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 809/1617] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.311899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.338128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807b0f2-3935-401f-8f4f-61834154b37e service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.342926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.344546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.347602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16a181-1d3c-4860-8d5a-ae2653ae976d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.349772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16a181-1d3c-4860-8d5a-ae2653ae976d service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.349994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16a181-1d3c-4860-8d5a-ae2653ae976d service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.352150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16a181-1d3c-4860-8d5a-ae2653ae976d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.356056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 809/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.357962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.371097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16a181-1d3c-4860-8d5a-ae2653ae976d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.373450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16a181-1d3c-4860-8d5a-ae2653ae976d service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.383021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f61c-be8e-4c64-9efc-72c096d37877 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.387017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 810/1619] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.387658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.387942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.388258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.392778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.393448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.393750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.394151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.412003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:08.413421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:08.419543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.421250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.425825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 811/1620] 10.222.0.49 () {58 vars in 1059 bytes} [Sun Jul 31 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:08.434484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.448112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 810/1621] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.474103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441de23d-298b-492d-91df-5cf4acdc085f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.496127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7215218-e257-4254-be2a-77bcef900f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.497801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7215218-e257-4254-be2a-77bcef900f94 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.498364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7215218-e257-4254-be2a-77bcef900f94 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.499053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7215218-e257-4254-be2a-77bcef900f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.513018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7215218-e257-4254-be2a-77bcef900f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.515234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7215218-e257-4254-be2a-77bcef900f94 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.527964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 812/1622] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.794407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:08.813976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75d49-c56a-455e-8f9a-4ddb89e1b839 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:08.826230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ec32e-24cb-4aa3-ad0e-cc215e7a8af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:08.827139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ec32e-24cb-4aa3-ad0e-cc215e7a8af9 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.827268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ec32e-24cb-4aa3-ad0e-cc215e7a8af9 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.827526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ec32e-24cb-4aa3-ad0e-cc215e7a8af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.843124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ec32e-24cb-4aa3-ad0e-cc215e7a8af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:08.846731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ec32e-24cb-4aa3-ad0e-cc215e7a8af9 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:08.867971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 811/1623] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:08.911586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6349a9e-d8fa-4d30-aae8-1d7674e58170 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.912109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6349a9e-d8fa-4d30-aae8-1d7674e58170 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.912658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6349a9e-d8fa-4d30-aae8-1d7674e58170 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.914876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 813/1624] 10.222.0.49 () {54 vars in 912 bytes} [Sun Jul 31 18:37:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:08.921921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.922865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.923148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.947263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:08.948530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:08.959661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd531c8-9669-4646-bd87-c931390ec5bf None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.959932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd531c8-9669-4646-bd87-c931390ec5bf None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.960205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd531c8-9669-4646-bd87-c931390ec5bf None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:08.964385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 814/1625] 10.222.0.49 () {54 vars in 912 bytes} [Sun Jul 31 18:37:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:37:08.970966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:08.971795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:08.972408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:09.068361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 812/1626] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:09.068361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:09.069032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:09.069032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:09.074005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 815/1627] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:09.084516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:09.184321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897e95-7c22-4d5f-9965-a2a4d767c386 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:09.189215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:09.189982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:09.190154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ba9f8-6d1d-4e75-97af-b0bcfdd070b7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:09.195679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:09.195679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:09.200732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717623aa-3a19-4d19-8a07-c4055ef89b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:09.202711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717623aa-3a19-4d19-8a07-c4055ef89b57 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:09.203741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717623aa-3a19-4d19-8a07-c4055ef89b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:09.204023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717623aa-3a19-4d19-8a07-c4055ef89b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:09.213958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:09.216900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:09.226381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717623aa-3a19-4d19-8a07-c4055ef89b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:09.228961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717623aa-3a19-4d19-8a07-c4055ef89b57 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:09.253901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 813/1629] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:09.260384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 816/1628] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:10.387365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:10.405351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03c48d-faf1-4a2f-9cd5-6c5270e9a2c0 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:10.416220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:10.417764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:10.419277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:10.419867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] PATH_INFO: `/v3/users/6e196cc129c342628a657636e0a3610f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:10.426071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e196cc129c342628a657636e0a3610f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:10.434093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:10.449993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:10.596257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e196cc129c342628a657636e0a3610f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:10.624814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e196cc129c342628a657636e0a3610f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:10.625505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e196cc129c342628a657636e0a3610f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:10.662945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e196cc129c342628a657636e0a3610f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:10.685215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e196cc129c342628a657636e0a3610f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:10.705785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 814/1630] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:10 2022] DELETE /identity/v3/users/6e196cc129c342628a657636e0a3610f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:10.712009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:10.713125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:10.713579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:10.747692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:10.749325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:10.898158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 817/1631] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:10.901895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:10.904333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:11.007977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248598e-30c6-4d5e-ba9d-79761f1ed811 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:11.011442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f575fc-52ad-47c9-a1cd-262114d187b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:11.012117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f575fc-52ad-47c9-a1cd-262114d187b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:11.012374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f575fc-52ad-47c9-a1cd-262114d187b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:11.012680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f575fc-52ad-47c9-a1cd-262114d187b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:11.033313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f575fc-52ad-47c9-a1cd-262114d187b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:11.034592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f575fc-52ad-47c9-a1cd-262114d187b2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:11.067257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 815/1632] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:12.404153 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:12.405877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:12.424202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b0cf-634d-4ffa-979c-e60654e2d76f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:12.441158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:12.441830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:12.442114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:12.442385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] PATH_INFO: `/v3/projects/df41262534584aa1930c9960bde9d2a8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:12.443621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df41262534584aa1930c9960bde9d2a8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:12.445679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:12.497529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:12.550104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df41262534584aa1930c9960bde9d2a8', 'request_id': 'req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:12.563677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df41262534584aa1930c9960bde9d2a8', 'request_id': 'req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:12.584240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df41262534584aa1930c9960bde9d2a8', 'request_id': 'req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:12.618860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] The token cache is being invalidate because project df41262534584aa1930c9960bde9d2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:12.619151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:12.622804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d15f7e4f-e8c1-4ea8-8db1-6e504b1ff622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:12.625763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 818/1633] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:12 2022] DELETE /identity/v3/projects/df41262534584aa1930c9960bde9d2a8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:12.631262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:12.631747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:12.631846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:12.669907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:12.671234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:12.821859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 816/1634] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:12.829937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:12.830745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:12.831045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:12.872473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:12.876896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:12.897693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 819/1635] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:12.900294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:12.918932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab01e19-a764-46dc-9ccf-fdbd6069c9ee None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:12.932917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:12.935874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:12.937202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:12.938485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:12.942694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:12.943988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:12.949348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:12.986036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 817/1636] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:12.990680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.003072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b45d2-9603-4a28-8e54-911ad0d58e9c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:13.010493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:13.011261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.011543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.011820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.012511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:13.015193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:13.053048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 820/1637] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:12 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:13.057401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.071391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32847a70-c882-4a0e-844f-30b9aac305e4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:13.112446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:13.198714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:13.199921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.200272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.200462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.201222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:13.214141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:13.260292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 818/1638] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:13 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:13.265198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.286462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14420655-3632-4ea4-b146-64c5b3af88e2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:13.305259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:13.307774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.308268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.309366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.311649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:13.323155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:13.367105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 821/1639] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:13.376727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.390585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4beb3-c9af-45c2-b211-79ecbf6dee48 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:13.400493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcccac-347b-4882-905b-435c1a32c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:13.401405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcccac-347b-4882-905b-435c1a32c76f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.401872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcccac-347b-4882-905b-435c1a32c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.402330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcccac-347b-4882-905b-435c1a32c76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.403258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcccac-347b-4882-905b-435c1a32c76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:13.405132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcccac-347b-4882-905b-435c1a32c76f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:13.433482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 819/1640] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:13.437539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.455181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e8c80-b025-43ab-9ebb-22421810526f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:13.462938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:13.463752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.464027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.464306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] PATH_INFO: `/v3/projects/e281870df3cc401d9876370c8c0b3cc5/users/be18e2793c0048c8abee48727a35bbae/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.491009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e281870df3cc401d9876370c8c0b3cc5, user_id=be18e2793c0048c8abee48727a35bbae, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:13.499187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:13.526627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 822/1641] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:13 2022] PUT /identity/v3/projects/e281870df3cc401d9876370c8c0b3cc5/users/be18e2793c0048c8abee48727a35bbae/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:13.538143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.538405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.538746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.598328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] MFA Rules not processed for user `be18e2793c0048c8abee48727a35bbae`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:13.599659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:13.717613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 820/1642] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:13.721845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.815628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e27cf-c83e-4136-87c2-28cb91dbaec4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:13.826654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:13.828503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:13.829206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:13.829576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:13.830657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:13.836021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:13.881458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 823/1643] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:13 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:13.884331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:13.903011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc5166-989c-4176-b60a-9d822579560c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:14.055945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:14.056490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.056912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.057360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:14.064051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:14.070081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:14.143013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 821/1644] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:13 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:14.147211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:14.165806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285374-8b38-4e78-a4e4-df1becd9456f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:14.178744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:14.180293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.181526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.181526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:14.185518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:14.189221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:14.219705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 824/1645] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:14.229070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:14.244990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f0ce4-7bbb-46b2-9ddc-721fac6a4e45 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:14.257738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:14.259509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.259793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.260079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] PATH_INFO: `/v3/projects/ef1ebc0a8e124ccebcbcd005308c875c/users/0c2be860a25c471aab2f0b15c84cdc78/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:14.285833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:14.293996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1ebc0a8e124ccebcbcd005308c875c, user_id=0c2be860a25c471aab2f0b15c84cdc78, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:14.297812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:14.335703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 822/1646] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:14 2022] PUT /identity/v3/projects/ef1ebc0a8e124ccebcbcd005308c875c/users/0c2be860a25c471aab2f0b15c84cdc78/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:14.335882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:14.356166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0070e-68d1-4074-9884-e2112d1c44f8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:14.409695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:14.496817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326f6dc-4149-4a63-b0a5-8b739053da71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:14.498137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f6dc-4149-4a63-b0a5-8b739053da71 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.498657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f6dc-4149-4a63-b0a5-8b739053da71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.499118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f6dc-4149-4a63-b0a5-8b739053da71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:14.499970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326f6dc-4149-4a63-b0a5-8b739053da71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:14.501941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326f6dc-4149-4a63-b0a5-8b739053da71 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:14.516116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 825/1647] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:14.520563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:14.542637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bea5b1-2a91-45cd-95c1-8f9ed82c6979 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:14.548093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:14.549320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.549608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.550028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] PATH_INFO: `/v3/projects/ef1ebc0a8e124ccebcbcd005308c875c/users/0c2be860a25c471aab2f0b15c84cdc78/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:14.552092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1ebc0a8e124ccebcbcd005308c875c, user_id=0c2be860a25c471aab2f0b15c84cdc78, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:14.555030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:14.580930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.581193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.581409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:14.584593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 823/1648] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:14 2022] PUT /identity/v3/projects/ef1ebc0a8e124ccebcbcd005308c875c/users/0c2be860a25c471aab2f0b15c84cdc78/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:14.633834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] MFA Rules not processed for user `0c2be860a25c471aab2f0b15c84cdc78`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:14.635412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:14.846504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 826/1649] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:14.857096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:14.859119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:14.949822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab524ba7-a477-4219-9f3b-4f85f0e5bce3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:14.955561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d8cd3-985d-4e30-830d-bdc0f54cbc6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:14.957955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d8cd3-985d-4e30-830d-bdc0f54cbc6c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:14.957955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d8cd3-985d-4e30-830d-bdc0f54cbc6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:14.957955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d8cd3-985d-4e30-830d-bdc0f54cbc6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:15.109297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d8cd3-985d-4e30-830d-bdc0f54cbc6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:15.114066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d8cd3-985d-4e30-830d-bdc0f54cbc6c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:15.142833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 824/1650] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:15.582863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:15.582863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:15.853362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d9d96-3f02-4071-960c-7e95c389ba97 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:15.864258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:15.865044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:15.865395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:15.865789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] PATH_INFO: `/v3/users/0b3e9aa34d474149a005ee8df63822fd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:15.868032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b3e9aa34d474149a005ee8df63822fd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:15.869211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: WARNING py.warnings [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:37:15.869211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: warnings.warn(msg) Jul 31 18:37:15.869211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: Jul 31 18:37:15.871408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:16.002435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:16.005009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b3e9aa34d474149a005ee8df63822fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:16.024859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b3e9aa34d474149a005ee8df63822fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:16.025795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b3e9aa34d474149a005ee8df63822fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:16.042864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b3e9aa34d474149a005ee8df63822fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:16.044619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b3e9aa34d474149a005ee8df63822fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:16.105019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 827/1651] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:15 2022] DELETE /identity/v3/users/0b3e9aa34d474149a005ee8df63822fd => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:16.127145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c633474-2c28-4f68-a568-fa8c39630092 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:16.127968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c633474-2c28-4f68-a568-fa8c39630092 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:16.128438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c633474-2c28-4f68-a568-fa8c39630092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:16.230695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0c633474-2c28-4f68-a568-fa8c39630092 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:16.252421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0c633474-2c28-4f68-a568-fa8c39630092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:16.407850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 825/1652] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:16.422792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:16.424516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:16.630772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0331-c171-4647-80ff-32953ba05aaf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:16.642116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68141012-b84b-40c1-946b-3e8bd2ae43ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:16.644030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141012-b84b-40c1-946b-3e8bd2ae43ef service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:16.644925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141012-b84b-40c1-946b-3e8bd2ae43ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:16.645842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68141012-b84b-40c1-946b-3e8bd2ae43ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:16.673027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68141012-b84b-40c1-946b-3e8bd2ae43ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:16.675563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68141012-b84b-40c1-946b-3e8bd2ae43ef service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:16.705166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 828/1653] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:17.222438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:17.223109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:17.223109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:17.308605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] MFA Rules not processed for user `e518e5691a7247de9d04ae9828809498`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:17.316510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:17.406883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:17.554544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 826/1654] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:17.555600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:17.556720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:17.557095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:17.672611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] MFA Rules not processed for user `e518e5691a7247de9d04ae9828809498`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:17.680317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:17.708319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 829/1655] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:17.720336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:17.724269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:18.070545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c8a8-8f77-409f-8329-7aa8f72a1833 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:18.084315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69653b5d-7bb2-4ff3-abfb-b68b42c3e05b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:18.084998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69653b5d-7bb2-4ff3-abfb-b68b42c3e05b service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:18.085184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69653b5d-7bb2-4ff3-abfb-b68b42c3e05b service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:18.090090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69653b5d-7bb2-4ff3-abfb-b68b42c3e05b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:18.110268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69653b5d-7bb2-4ff3-abfb-b68b42c3e05b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:18.112498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69653b5d-7bb2-4ff3-abfb-b68b42c3e05b service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:18.173181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 827/1656] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:18.200126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:18.202884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:18.259521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413e87f-b141-4d53-9fb9-b08da8d95288 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:18.266115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:18.266893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:18.267413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:18.267863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] PATH_INFO: `/v3/users/3aaaec3395b04183926746825115ec62` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:18.269635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aaaec3395b04183926746825115ec62)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:18.271775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:18.381609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aaaec3395b04183926746825115ec62'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:18.393597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aaaec3395b04183926746825115ec62'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:18.394623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aaaec3395b04183926746825115ec62'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:18.418004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aaaec3395b04183926746825115ec62'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:18.418981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aaaec3395b04183926746825115ec62'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:18.452904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 830/1657] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:18 2022] DELETE /identity/v3/users/3aaaec3395b04183926746825115ec62 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:18.458296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a83fc-6d61-43e3-9f04-8011ab9f1441 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:18.459304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a83fc-6d61-43e3-9f04-8011ab9f1441 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:18.459808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a83fc-6d61-43e3-9f04-8011ab9f1441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:18.483732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-893a83fc-6d61-43e3-9f04-8011ab9f1441 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:18.485327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-893a83fc-6d61-43e3-9f04-8011ab9f1441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:18.603488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 828/1658] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:18.643541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:18.647031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:18.719287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:18.763767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e512924-e3d7-4e8a-825e-b9a0b317a4d0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:18.775437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e4c68-7cef-448e-97df-2d5a0f4dec5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:18.776849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e4c68-7cef-448e-97df-2d5a0f4dec5b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:18.777364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e4c68-7cef-448e-97df-2d5a0f4dec5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:18.777894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e4c68-7cef-448e-97df-2d5a0f4dec5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:18.789622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e4c68-7cef-448e-97df-2d5a0f4dec5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:18.791720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e4c68-7cef-448e-97df-2d5a0f4dec5b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:18.805132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 831/1659] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:19.648527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d104c5-e4a2-46bc-9249-eb4a958bd5d0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:19.649653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d104c5-e4a2-46bc-9249-eb4a958bd5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:19.650128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d104c5-e4a2-46bc-9249-eb4a958bd5d0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:19.652687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 829/1660] 10.222.0.49 () {54 vars in 912 bytes} [Sun Jul 31 18:37:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:19.670324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:19.670647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:19.670962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:19.704357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d104c5-e4a2-46bc-9249-eb4a958bd5d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:19.706187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-54d104c5-e4a2-46bc-9249-eb4a958bd5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:19.742472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d104c5-e4a2-46bc-9249-eb4a958bd5d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:19.756162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:19.757017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:19.757422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:19.757990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] PATH_INFO: `/v3/projects/5737a28d766849f8ab641c4594ed46f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:19.759945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5737a28d766849f8ab641c4594ed46f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:19.762759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 18:37:19.762759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jul 31 18:37:19.762759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: Jul 31 18:37:19.770680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:19.776634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:19.779699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:19.804937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 832/1661] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:37:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:19.905330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:19.941561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5737a28d766849f8ab641c4594ed46f0', 'request_id': 'req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:19.974594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:19.977121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5737a28d766849f8ab641c4594ed46f0', 'request_id': 'req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:19.977313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:20.002338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5737a28d766849f8ab641c4594ed46f0', 'request_id': 'req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.005488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae6e85-962c-429f-ad8b-020c5d0f2425 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:20.038393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] The token cache is being invalidate because project 5737a28d766849f8ab641c4594ed46f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:20.039044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.040307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e5e2a3b2-d33d-41fe-ad3b-89ea69e3dc08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.045888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 830/1662] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:19 2022] DELETE /identity/v3/projects/5737a28d766849f8ab641c4594ed46f0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:20.050932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.051574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.052054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.103419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:20.106988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:20.182388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:20.183435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.183930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.184274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.192406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:20.214098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:20.215801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1cb9b-c344-4f4e-9cc3-9dca307b363f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:20.252496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 833/1663] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:20.276897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 831/1664] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:20.287161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.287446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.287797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.364629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:20.365902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:20.393195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 834/1665] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:20.394550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:20.410370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abcbbb-7a3d-4920-a78b-abc9a91f4b82 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:20.422374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:20.423321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.423771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.424213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.425178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:20.427006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:20.428854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:20.464708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 832/1666] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:20.470264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:20.483075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc68c03-e179-4ada-8662-af62d7509649 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:20.494769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:20.495564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.495928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.496114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.499541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:20.504868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:20.539810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:20.541610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:20.570444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd09eb4-fae2-43d1-8b50-cfee9dbd7777 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:20.577644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:20.578315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.578593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.578863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] PATH_INFO: `/v3/projects/c36ffef5602f4ffd806ae72e49d3b339` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.581067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 835/1667] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:20 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:20.581261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36ffef5602f4ffd806ae72e49d3b339)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:20.581965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:20.583264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:20.709402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36ffef5602f4ffd806ae72e49d3b339', 'request_id': 'req-00ded819-74c5-4f7b-b41d-746de197791e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.725333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36ffef5602f4ffd806ae72e49d3b339', 'request_id': 'req-00ded819-74c5-4f7b-b41d-746de197791e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.741424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36ffef5602f4ffd806ae72e49d3b339', 'request_id': 'req-00ded819-74c5-4f7b-b41d-746de197791e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.786474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] The token cache is being invalidate because project c36ffef5602f4ffd806ae72e49d3b339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:20.786763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.788820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-00ded819-74c5-4f7b-b41d-746de197791e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:20.796791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 833/1668] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:20 2022] DELETE /identity/v3/projects/c36ffef5602f4ffd806ae72e49d3b339 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:20.796990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff119e6-4f9a-4404-bbc9-f3c3e868f86a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:20.798298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69df91f-b71b-42d0-9017-09880eb34a58 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.798579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69df91f-b71b-42d0-9017-09880eb34a58 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.798849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69df91f-b71b-42d0-9017-09880eb34a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.841207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a69df91f-b71b-42d0-9017-09880eb34a58 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:20.842704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a69df91f-b71b-42d0-9017-09880eb34a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:20.870193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:20.908296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:20.950941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:20.951996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.952470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.952968 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:20.954736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:20.960851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 834/1669] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:20.963346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:20.966531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:20.966814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:20.967119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.014154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:21.015322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:21.017224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 836/1670] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:20 2022] POST /identity/v3/users => generated 303 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:21.022192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.024680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:21.038427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 835/1671] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:21.039704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.082976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425bac-529f-4de2-82ed-d8ebbca48687 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.094855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.095519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.096158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.096435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.097207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:21.097511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.099168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.104312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15e82-d14b-4cdf-84b2-a6b901725fab admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.113206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.113859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.114137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.114409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.115124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.125561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.141924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 836/1672] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:21.144499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.160644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 837/1673] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:21.168704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.169943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee24a-4447-41a5-8f83-847998a9554d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.180039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.180697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.181005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.181277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.181956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.192447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.233290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa9b4a-3680-4d2a-bb36-2f38cf4bc108 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.247397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.248556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.249069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.249601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.250507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.252337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.273779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 837/1674] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:21 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:21.274856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.278499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:21.283653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 838/1675] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:21.284884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.303525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7236f8-0b81-4641-b5a2-2bcb031be034 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.444615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fbe2a-6e21-4b92-852d-7ae6e446936f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.464958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.465124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.465758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.466035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.466135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.466307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.466722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.466984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.467308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.469806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.470065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:21.523702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 839/1676] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:21 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:21.530681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.554581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a608-9316-4e60-81ea-96ac3ace38ba admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.569061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420cef0-3f6b-4d9c-84ea-db08f150fcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.574134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420cef0-3f6b-4d9c-84ea-db08f150fcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.574134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420cef0-3f6b-4d9c-84ea-db08f150fcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.574134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420cef0-3f6b-4d9c-84ea-db08f150fcfb admin admin] PATH_INFO: `/v3/projects/bfadaf0642c842b186fa08c35d9dcefe/users/387f455d715a49d6b36daffd7d65cce7/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.603796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.605903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.607740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420cef0-3f6b-4d9c-84ea-db08f150fcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfadaf0642c842b186fa08c35d9dcefe, user_id=387f455d715a49d6b36daffd7d65cce7, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.613024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420cef0-3f6b-4d9c-84ea-db08f150fcfb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.625334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 838/1677] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:21.627404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.649017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 840/1678] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:21 2022] PUT /identity/v3/projects/bfadaf0642c842b186fa08c35d9dcefe/users/387f455d715a49d6b36daffd7d65cce7/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:21.654294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.654892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.655353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.659303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22160-b0f9-4f4d-94fa-f7df2c3edf0f service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.696253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:21.700305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:21.771648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:21.820621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.840747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.841256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.841663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.849196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.851748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.869771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 839/1679] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:21.871087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.902346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efb85d-b269-4b91-a65e-97485c5cdefe admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.915916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 841/1680] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:21.921914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.922644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:21.922998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:21.923892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:21.924686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:21.926462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:21.948130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.949381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 840/1681] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:21.960713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:21.963887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd932e5-1374-4884-997a-ba72e1ea9655 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:21.968422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:21.993740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515fa1e-9026-472b-922f-71ef6500d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:21.998531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515fa1e-9026-472b-922f-71ef6500d827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:22.004720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515fa1e-9026-472b-922f-71ef6500d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:22.005257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515fa1e-9026-472b-922f-71ef6500d827 admin admin] PATH_INFO: `/v3/projects/4910d38f8e3141c6895f9f9dc3e10834/users/fb03fd5a46e54d06baa81e38679e3134/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:22.008530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:22.054115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515fa1e-9026-472b-922f-71ef6500d827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4910d38f8e3141c6895f9f9dc3e10834, user_id=fb03fd5a46e54d06baa81e38679e3134, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:22.062319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515fa1e-9026-472b-922f-71ef6500d827 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:22.106209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 842/1682] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:21 2022] PUT /identity/v3/projects/4910d38f8e3141c6895f9f9dc3e10834/users/fb03fd5a46e54d06baa81e38679e3134/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:22.118240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:22.118524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:22.118802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:22.142153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22fcc9-e16d-4048-a0ec-b690734fa4aa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:22.220859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] MFA Rules not processed for user `fb03fd5a46e54d06baa81e38679e3134`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:22.222287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:22.265311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec38596-2e82-419a-834d-e084c956dfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:22.267235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38596-2e82-419a-834d-e084c956dfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:22.270537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38596-2e82-419a-834d-e084c956dfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:22.271221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38596-2e82-419a-834d-e084c956dfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:22.476680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 843/1683] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:22.478776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec38596-2e82-419a-834d-e084c956dfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:22.484022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec38596-2e82-419a-834d-e084c956dfa6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:22.488234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:22.490786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:22.546192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 841/1684] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:22.683106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6c889-d79d-4f1a-9513-fa979f2b270a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:22.701806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9b968-543d-4864-811f-d265ef9ccf9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:22.702863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b968-543d-4864-811f-d265ef9ccf9a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:22.703270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b968-543d-4864-811f-d265ef9ccf9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:22.703769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b968-543d-4864-811f-d265ef9ccf9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:22.715910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9b968-543d-4864-811f-d265ef9ccf9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:22.727652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9b968-543d-4864-811f-d265ef9ccf9a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:22.762434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 844/1685] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:23.058315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ba27-1229-408a-8f65-d18e4db990bf None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:23.058783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ba27-1229-408a-8f65-d18e4db990bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:23.058902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ba27-1229-408a-8f65-d18e4db990bf None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:23.061563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 842/1686] 10.222.0.49 () {54 vars in 961 bytes} [Sun Jul 31 18:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:23.068992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998a133-6c0e-4f9f-981e-bbe3e24c4ab7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:23.069871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998a133-6c0e-4f9f-981e-bbe3e24c4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:23.070323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998a133-6c0e-4f9f-981e-bbe3e24c4ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:23.119385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f998a133-6c0e-4f9f-981e-bbe3e24c4ab7 None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:23.120952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f998a133-6c0e-4f9f-981e-bbe3e24c4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:23.308889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 845/1687] 10.222.0.49 () {58 vars in 1058 bytes} [Sun Jul 31 18:37:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:23.317494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ba27-1229-408a-8f65-d18e4db990bf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:23.366733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ba27-1229-408a-8f65-d18e4db990bf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:23.376720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:23.377602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:23.378119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:23.378544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:23.385352 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:23.462258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:23.464087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:23.465606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a353da-dd8a-41cf-af74-ddbb2f627606 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:23.466414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a353da-dd8a-41cf-af74-ddbb2f627606 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:23.466700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a353da-dd8a-41cf-af74-ddbb2f627606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:23.512688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:23.536182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 843/1688] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:23.543394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-68a353da-dd8a-41cf-af74-ddbb2f627606 None None] MFA Rules not processed for user `120c06d6dee6425ea5366b363d5a5f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:23.544937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-68a353da-dd8a-41cf-af74-ddbb2f627606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:23.846457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 846/1689] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:23.865912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:23.869942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:23.940139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:24.088734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3254f2c-f666-49c4-bfbb-d79698a88837 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:24.089362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3254f2c-f666-49c4-bfbb-d79698a88837 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:24.089362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3254f2c-f666-49c4-bfbb-d79698a88837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:24.132003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aaabaa-326f-4e0e-8c8a-e170fd441af5 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:24.161492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:24.162164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:24.162444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:24.162715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:24.190706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:24.195845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f3254f2c-f666-49c4-bfbb-d79698a88837 None None] MFA Rules not processed for user `4dc88d9b0bc743119dcb7dc2fa8fdcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:24.196722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:24.200369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f3254f2c-f666-49c4-bfbb-d79698a88837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:24.237662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 844/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:24.506641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 847/1691] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:24.521621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:24.542401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bdb9e-9fa2-4492-8bd3-0fb96559558d service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:24.559087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:24.559781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:24.560060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:24.560338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:24.580223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:24.582059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:24.606281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 845/1692] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:26.849756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:26.850231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:26.850801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:26.963786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] MFA Rules not processed for user `387f455d715a49d6b36daffd7d65cce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:26.965127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:27.112130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 848/1693] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:27.116498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.131751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bb307-e8e3-4cca-a537-bd0153ef0c1e service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:27.144716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a3a0f-7cbe-42e0-b59c-f489b544d78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:27.145705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3a0f-7cbe-42e0-b59c-f489b544d78d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.146184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3a0f-7cbe-42e0-b59c-f489b544d78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.146665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3a0f-7cbe-42e0-b59c-f489b544d78d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.148093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a3a0f-7cbe-42e0-b59c-f489b544d78d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:27.150827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a3a0f-7cbe-42e0-b59c-f489b544d78d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:27.200019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 846/1694] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:27 2022] POST /identity/v3/projects => generated 405 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:27.203993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.221870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8c809-4642-46e3-867f-71f988e72ff1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:27.267963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:27.330833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.331123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.331403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.348806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f3026-3e7b-473c-a44e-4ff8e982f425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:27.350883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3026-3e7b-473c-a44e-4ff8e982f425 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.351235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3026-3e7b-473c-a44e-4ff8e982f425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.352495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3026-3e7b-473c-a44e-4ff8e982f425 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.354666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3026-3e7b-473c-a44e-4ff8e982f425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:27.358502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3026-3e7b-473c-a44e-4ff8e982f425 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:27.410354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] MFA Rules not processed for user `88472fa130654657af599106c6aac279`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:27.413320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:27.417569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 849/1695] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:27 2022] POST /identity/v3/users => generated 309 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:27.418825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.421196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.423284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.508754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:27.510176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:27.579658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 847/1696] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:27.580673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.585577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 850/1697] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:37:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:27.592183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.597236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:27.610336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ed68b-8c9d-4cf9-b0a7-6f1e5143d6f0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:27.625450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:27.626122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.626372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.626733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.627441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:27.629450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:27.663124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 848/1698] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:27.665302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.690131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c50b0e-8659-4057-b93d-55f58b9398ef admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:27.720454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7e130-728a-4e16-b3f0-b270127e08da None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:27.726752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:27.727546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.727827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.728107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] PATH_INFO: `/v3/projects/205ad8f793e54c2b9478686413215303/users/552948580d4645469bdf61ba968782a9/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.732854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:27.733654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:27.733810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:27.734083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:27.777014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:27.778976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:27.788650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=205ad8f793e54c2b9478686413215303, user_id=552948580d4645469bdf61ba968782a9, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:27.795718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:27.826745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 851/1699] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:27.841455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 849/1700] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:27 2022] PUT /identity/v3/projects/205ad8f793e54c2b9478686413215303/users/552948580d4645469bdf61ba968782a9/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:27.845620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.874923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fa7b8-8856-4824-9141-df93d0b48e1a service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:27.904112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:27.960739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:27.963344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:28.108289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:28.166828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:28.167484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:28.167761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:28.168039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:28.179408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:28.186602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:28.243621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 852/1701] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:37:28.245259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:28.287882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1919eb-e2ef-437b-b1ab-b906a2ab8784 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:28.301105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e909bf-7bba-42d4-b723-c9a6a0243f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:28.303242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e909bf-7bba-42d4-b723-c9a6a0243f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:28.303242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e909bf-7bba-42d4-b723-c9a6a0243f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:28.303736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e909bf-7bba-42d4-b723-c9a6a0243f8c admin admin] PATH_INFO: `/v3/projects/205ad8f793e54c2b9478686413215303/users/552948580d4645469bdf61ba968782a9/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:28.310261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e909bf-7bba-42d4-b723-c9a6a0243f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=205ad8f793e54c2b9478686413215303, user_id=552948580d4645469bdf61ba968782a9, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:28.317958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e909bf-7bba-42d4-b723-c9a6a0243f8c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:28.328135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68608257-74c6-4cc1-aa5c-a9c398b9bdaa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:28.353803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 853/1702] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:28 2022] PUT /identity/v3/projects/205ad8f793e54c2b9478686413215303/users/552948580d4645469bdf61ba968782a9/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:28.354144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01795944-85cc-4e58-91da-1bbace441376 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:28.355328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795944-85cc-4e58-91da-1bbace441376 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:28.356617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795944-85cc-4e58-91da-1bbace441376 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:28.356617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795944-85cc-4e58-91da-1bbace441376 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:28.433907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795944-85cc-4e58-91da-1bbace441376 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:28.437201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795944-85cc-4e58-91da-1bbace441376 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:28.455879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 850/1703] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:37:30.283552 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ffc46-651c-40a6-8252-179aa431792d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:30.284007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ffc46-651c-40a6-8252-179aa431792d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:30.284007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ffc46-651c-40a6-8252-179aa431792d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:30.377728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-984ffc46-651c-40a6-8252-179aa431792d None None] MFA Rules not processed for user `0d7ed577305d48c795f280e49b438bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:30.380723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-984ffc46-651c-40a6-8252-179aa431792d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:30.519205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 854/1704] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:30.530027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28fdfc-6458-4c20-94e6-f27d49a1df70 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:30.530934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28fdfc-6458-4c20-94e6-f27d49a1df70 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:30.531391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28fdfc-6458-4c20-94e6-f27d49a1df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:30.570853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6c28fdfc-6458-4c20-94e6-f27d49a1df70 None None] MFA Rules not processed for user `0d7ed577305d48c795f280e49b438bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:30.572328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6c28fdfc-6458-4c20-94e6-f27d49a1df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:30.595491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 851/1705] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:30.606621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ffc46-651c-40a6-8252-179aa431792d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:30.760238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ffc46-651c-40a6-8252-179aa431792d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:30.776720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:30.777607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:30.778026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:30.778392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:30.798880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:30.800602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:30.821975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 855/1706] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:31.365970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785f247-fdf2-470e-805f-fb05fb5b030a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:31.366312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785f247-fdf2-470e-805f-fb05fb5b030a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:31.366589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785f247-fdf2-470e-805f-fb05fb5b030a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:31.412537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d785f247-fdf2-470e-805f-fb05fb5b030a None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:31.417916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d785f247-fdf2-470e-805f-fb05fb5b030a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:31.558841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 852/1707] 10.222.0.57 () {58 vars in 1015 bytes} [Sun Jul 31 18:37:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:31.610850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:31.612466 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:31.626938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:31.636564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee549aa-8fcf-4757-9158-bbddedae90a1 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:31.648185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5f6e3-f462-473a-bf57-d0d734717e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:31.649168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5f6e3-f462-473a-bf57-d0d734717e67 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:31.649602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5f6e3-f462-473a-bf57-d0d734717e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:31.650122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5f6e3-f462-473a-bf57-d0d734717e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:31.667102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5f6e3-f462-473a-bf57-d0d734717e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:31.668485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5f6e3-f462-473a-bf57-d0d734717e67 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:31.691299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 856/1708] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:32.038062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3a078-8494-4e13-8a08-178f9c5a87c4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:32.038585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3a078-8494-4e13-8a08-178f9c5a87c4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:32.039011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3a078-8494-4e13-8a08-178f9c5a87c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:32.077310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-54e3a078-8494-4e13-8a08-178f9c5a87c4 None None] MFA Rules not processed for user `120c06d6dee6425ea5366b363d5a5f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:32.088892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-54e3a078-8494-4e13-8a08-178f9c5a87c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:32.271520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:32.302985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 853/1709] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:32.476424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2882d-7526-4d08-82aa-cc1ad58124ef None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:32.476734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2882d-7526-4d08-82aa-cc1ad58124ef None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:32.477246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2882d-7526-4d08-82aa-cc1ad58124ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:32.534921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-91c2882d-7526-4d08-82aa-cc1ad58124ef None None] MFA Rules not processed for user `4dc88d9b0bc743119dcb7dc2fa8fdcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:32.537106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-91c2882d-7526-4d08-82aa-cc1ad58124ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:32.726680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 857/1710] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:34.112495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5c2c8-b7c8-40fa-9352-391c39834804 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:34.113927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5c2c8-b7c8-40fa-9352-391c39834804 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:34.114380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5c2c8-b7c8-40fa-9352-391c39834804 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:34.116366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 854/1711] 10.222.0.49 () {54 vars in 912 bytes} [Sun Jul 31 18:37:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:34.128668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff79ec-b44a-4b1e-afce-8877c2b70345 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:34.129931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff79ec-b44a-4b1e-afce-8877c2b70345 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:34.130379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff79ec-b44a-4b1e-afce-8877c2b70345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:34.177175 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-33ff79ec-b44a-4b1e-afce-8877c2b70345 None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:34.180152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-33ff79ec-b44a-4b1e-afce-8877c2b70345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:34.194932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 858/1712] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:37:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:34.229025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5c2c8-b7c8-40fa-9352-391c39834804 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:34.255665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201562-cb14-49b4-ba0d-686a471575fc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:34.256208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201562-cb14-49b4-ba0d-686a471575fc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:34.256657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5c2c8-b7c8-40fa-9352-391c39834804 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:34.257097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201562-cb14-49b4-ba0d-686a471575fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:34.266349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153332b0-15f6-4ad4-9a20-27b2484dbfaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:34.271451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153332b0-15f6-4ad4-9a20-27b2484dbfaa service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:34.271743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153332b0-15f6-4ad4-9a20-27b2484dbfaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:34.272996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153332b0-15f6-4ad4-9a20-27b2484dbfaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:34.299492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153332b0-15f6-4ad4-9a20-27b2484dbfaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:34.301350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153332b0-15f6-4ad4-9a20-27b2484dbfaa service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:34.334843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 855/1713] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:34.346245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-88201562-cb14-49b4-ba0d-686a471575fc None None] MFA Rules not processed for user `552948580d4645469bdf61ba968782a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:34.347761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-88201562-cb14-49b4-ba0d-686a471575fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:34.505355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 859/1714] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:34 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:35.394399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d3109-0788-45ff-af49-489bb01c92aa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:35.395434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d3109-0788-45ff-af49-489bb01c92aa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:35.395994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d3109-0788-45ff-af49-489bb01c92aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:35.445881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-db3d3109-0788-45ff-af49-489bb01c92aa None None] MFA Rules not processed for user `88472fa130654657af599106c6aac279`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:35.452861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db3d3109-0788-45ff-af49-489bb01c92aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:35.723729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 856/1715] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:36.274958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee148-e257-4e00-a0e5-640c0834063e None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:36.275506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee148-e257-4e00-a0e5-640c0834063e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:36.275506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee148-e257-4e00-a0e5-640c0834063e None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:36.276867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 860/1716] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:36.486273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868475b8-8b6e-4d55-824b-cb8625ff5869 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:36.487238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868475b8-8b6e-4d55-824b-cb8625ff5869 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:36.487735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868475b8-8b6e-4d55-824b-cb8625ff5869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:36.535988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-868475b8-8b6e-4d55-824b-cb8625ff5869 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:36.542244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-868475b8-8b6e-4d55-824b-cb8625ff5869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:36.588830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 857/1717] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:37:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:36.889025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeae34-b67d-426c-acec-0800aea7b69d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:36.889722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeae34-b67d-426c-acec-0800aea7b69d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:36.890181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeae34-b67d-426c-acec-0800aea7b69d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:36.892056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 861/1718] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:37:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:37:36.906242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce94449-d2dc-407e-8dd6-b78c195b4748 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:36.907246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce94449-d2dc-407e-8dd6-b78c195b4748 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:36.907663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce94449-d2dc-407e-8dd6-b78c195b4748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:36.965508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9ce94449-d2dc-407e-8dd6-b78c195b4748 None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:36.970777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ce94449-d2dc-407e-8dd6-b78c195b4748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:37.194542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 858/1719] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:37:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:37.271486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eeae34-b67d-426c-acec-0800aea7b69d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:37.299721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eeae34-b67d-426c-acec-0800aea7b69d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:37.310423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:37.311108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:37.311379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:37.311655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:37.331501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:37.333179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:37.369073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 862/1720] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:37.513396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:37.514472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:37.514893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:37.573994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] MFA Rules not processed for user `0d7ed577305d48c795f280e49b438bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:37.576134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:37.601806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 859/1721] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:37.976013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:37.998939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27739a7f-4e47-4206-a309-e98ca0cb5a42 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:38.021409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:38.022171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:38.022558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:38.022957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:38.055504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:38.057181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:38.094031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 863/1722] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:38.929967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:39.117542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7baae2-a820-476b-a520-4e84c06fdbb8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:39.132518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:39.133241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:39.133522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:39.133796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] PATH_INFO: `/v3/users/387f455d715a49d6b36daffd7d65cce7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:39.138180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=387f455d715a49d6b36daffd7d65cce7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:39.141031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:39.176910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:39.264173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387f455d715a49d6b36daffd7d65cce7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:39.287451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387f455d715a49d6b36daffd7d65cce7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:39.287996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387f455d715a49d6b36daffd7d65cce7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:39.314906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387f455d715a49d6b36daffd7d65cce7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:39.331789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387f455d715a49d6b36daffd7d65cce7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:39.364288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 860/1723] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:38 2022] DELETE /identity/v3/users/387f455d715a49d6b36daffd7d65cce7 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:40.437125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:40.456228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0a6d4-d616-4f40-a801-35dedcb1f853 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:40.622610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:40.670040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:40.671438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:40.671910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:40.672342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] PATH_INFO: `/v3/projects/bfadaf0642c842b186fa08c35d9dcefe` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:40.675289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfadaf0642c842b186fa08c35d9dcefe)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:40.681411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:40.807541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfadaf0642c842b186fa08c35d9dcefe', 'request_id': 'req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:40.824063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfadaf0642c842b186fa08c35d9dcefe', 'request_id': 'req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:40.840311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfadaf0642c842b186fa08c35d9dcefe', 'request_id': 'req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:40.868610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] The token cache is being invalidate because project bfadaf0642c842b186fa08c35d9dcefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:40.868965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:40.871263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:40.876470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 864/1724] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:40 2022] DELETE /identity/v3/projects/bfadaf0642c842b186fa08c35d9dcefe => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:40.880842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:40.882477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:40.904519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768b80a-1d6d-46c8-ad4c-ff19cdfec7b9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:41.047675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:41.048534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:41.049323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:41.050173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] PATH_INFO: `/v3/users/552948580d4645469bdf61ba968782a9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:41.051663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=552948580d4645469bdf61ba968782a9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:41.053645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:41.152108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552948580d4645469bdf61ba968782a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.175820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552948580d4645469bdf61ba968782a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.177183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552948580d4645469bdf61ba968782a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.196397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552948580d4645469bdf61ba968782a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.204428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-435f5d1d-21bc-4cd5-89fa-66c38188ba8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552948580d4645469bdf61ba968782a9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.214162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 861/1725] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:40 2022] DELETE /identity/v3/users/552948580d4645469bdf61ba968782a9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:41.459581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:41.462135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:41.484301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb11b8-d4c3-4756-90e5-54f025e22f0f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:41.563650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:41.564391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:41.564742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:41.565045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] PATH_INFO: `/v3/projects/205ad8f793e54c2b9478686413215303` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:41.566344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=205ad8f793e54c2b9478686413215303)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:41.568279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:41.622508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205ad8f793e54c2b9478686413215303', 'request_id': 'req-d1044e9f-062d-448a-9a28-cc8d63f86148'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.628779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205ad8f793e54c2b9478686413215303', 'request_id': 'req-d1044e9f-062d-448a-9a28-cc8d63f86148'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.635197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205ad8f793e54c2b9478686413215303', 'request_id': 'req-d1044e9f-062d-448a-9a28-cc8d63f86148'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.655347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] The token cache is being invalidate because project 205ad8f793e54c2b9478686413215303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:41.655655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.657782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1044e9f-062d-448a-9a28-cc8d63f86148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:41.664299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 865/1726] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:41 2022] DELETE /identity/v3/projects/205ad8f793e54c2b9478686413215303 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:41.666282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:41.666661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:41.667074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:41.690669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:41.692035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:41.700794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:41.767681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 862/1727] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:41.775676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:41.776138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:41.776505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:41.800553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:41.802488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:41.827860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 866/1728] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:41.834228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:41.847725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4a3f0-a42c-4e36-bb03-3808594c95f7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:41.854133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:41.854832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:41.855117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:41.855421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:41.856207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:41.856452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:41.858291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:41.871504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 863/1729] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:41.876103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:41.881140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:41.886639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0134eaa-66e6-4425-a04c-709e7581b38c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:41.893688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:41.895028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:41.895350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:41.895898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:41.897115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:41.899176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:41.933685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 867/1730] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:41 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:41.939328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:41.951748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c3c71-92f4-4a32-a8cc-e375cfff0648 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.039310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.039981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.040323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.040599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.041442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.044207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.081385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 864/1731] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:41 2022] POST /identity/v3/users => generated 281 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:42.085555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.097343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeec40-f35f-47ff-84cb-30cc9916b32f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.103124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.103746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.104011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.104303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.105015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.106668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.119916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 868/1732] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:42.125765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.137331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe34ca6-22b3-4040-9cdc-80ddba93ce1a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.168867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f317b-81c9-4e9b-af52-92801bf4e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.169928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f317b-81c9-4e9b-af52-92801bf4e64b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.170385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f317b-81c9-4e9b-af52-92801bf4e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.170896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f317b-81c9-4e9b-af52-92801bf4e64b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.171996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f317b-81c9-4e9b-af52-92801bf4e64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.179940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f317b-81c9-4e9b-af52-92801bf4e64b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.226365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 865/1733] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:42.230632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.241283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afad16-7c31-48a2-a24d-a0455718f747 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.246399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.247159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.247406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.247679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] PATH_INFO: `/v3/projects/66ce5d5d6c3d45d19ecc15e178eb1adb/users/40cac066aa934b1bb4880bb055951209/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.266487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ce5d5d6c3d45d19ecc15e178eb1adb, user_id=40cac066aa934b1bb4880bb055951209, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.274573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.302867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 869/1734] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:42 2022] PUT /identity/v3/projects/66ce5d5d6c3d45d19ecc15e178eb1adb/users/40cac066aa934b1bb4880bb055951209/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:42.315119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.315399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.315683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.353479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] MFA Rules not processed for user `40cac066aa934b1bb4880bb055951209`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:42.354721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:42.449855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 866/1735] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:42.455490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.539472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68484079-eca3-4641-8d8d-0836f5398135 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.549910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.550562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.550840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.551161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.551791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.556914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.595972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 870/1736] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:42 2022] POST /identity/v3/projects => generated 363 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:42.600075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.612904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f4938-9f54-48c3-b55d-f444f0f064f2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.618444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:42.699234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.700131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.700440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.701127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.702240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.704958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.746870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 867/1737] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:42 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:42.750587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.765684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66fbed-2c33-45f3-abfe-cd57d9aeffd0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.771386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.777336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.777945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.778395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.779562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.781791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.799801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 871/1738] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:42.804495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.824530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13e292-412e-49a3-83b1-9e95160f8650 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.830605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.831580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.832024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.832565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] PATH_INFO: `/v3/projects/96d4213d17c944cdb6a7f17d65adecd2/users/936cf329c85746d1a0edaf8e737667f5/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.852439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d4213d17c944cdb6a7f17d65adecd2, user_id=936cf329c85746d1a0edaf8e737667f5, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.855548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.877773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 868/1739] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:42 2022] PUT /identity/v3/projects/96d4213d17c944cdb6a7f17d65adecd2/users/936cf329c85746d1a0edaf8e737667f5/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:42.882024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:42.892716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c27928-0052-4841-b435-54bd9963f33a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:42.976125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9c89-0d48-4a15-bde5-cc55b6e6087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:42.976640 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9c89-0d48-4a15-bde5-cc55b6e6087a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:42.977235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9c89-0d48-4a15-bde5-cc55b6e6087a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:42.977506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9c89-0d48-4a15-bde5-cc55b6e6087a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:42.978280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9c89-0d48-4a15-bde5-cc55b6e6087a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:42.980370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9c89-0d48-4a15-bde5-cc55b6e6087a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:42.993360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 872/1740] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:37:42.996977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:43.010006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3af0b-508d-4475-a1e5-f7283e939c27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:43.015300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:43.016012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:43.016286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:43.016565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] PATH_INFO: `/v3/projects/96d4213d17c944cdb6a7f17d65adecd2/users/936cf329c85746d1a0edaf8e737667f5/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:43.018758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d4213d17c944cdb6a7f17d65adecd2, user_id=936cf329c85746d1a0edaf8e737667f5, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:43.021509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:43.043667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 869/1741] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:42 2022] PUT /identity/v3/projects/96d4213d17c944cdb6a7f17d65adecd2/users/936cf329c85746d1a0edaf8e737667f5/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:43.053789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:43.054441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:43.054858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:43.125653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] MFA Rules not processed for user `936cf329c85746d1a0edaf8e737667f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:43.126949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:43.131528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:43.306486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 873/1742] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:43.320556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:43.322668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:43.410175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cc826-a4a8-4d06-876d-4a98bf9905c0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:43.415684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ce955-5c31-49f1-ab38-83524f76e055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:43.416413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ce955-5c31-49f1-ab38-83524f76e055 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:43.416820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ce955-5c31-49f1-ab38-83524f76e055 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:43.417189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ce955-5c31-49f1-ab38-83524f76e055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:43.431232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ce955-5c31-49f1-ab38-83524f76e055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:43.432754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ce955-5c31-49f1-ab38-83524f76e055 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:43.443215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 870/1743] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:44.011279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:44.078305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf08b99-725c-48a6-ad28-1b1343be8f77 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:44.083717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:44.084414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:44.084727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:44.085011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] PATH_INFO: `/v3/users/40cac066aa934b1bb4880bb055951209` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:44.086485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40cac066aa934b1bb4880bb055951209)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:44.088651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:44.170616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40cac066aa934b1bb4880bb055951209'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:44.181931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40cac066aa934b1bb4880bb055951209'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:44.182877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40cac066aa934b1bb4880bb055951209'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:44.196319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40cac066aa934b1bb4880bb055951209'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:44.197080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40cac066aa934b1bb4880bb055951209'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:44.289828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 874/1744] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:44 2022] DELETE /identity/v3/users/40cac066aa934b1bb4880bb055951209 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:44.291651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:44.292127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:44.292459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:44.317829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:44.320179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:44.415210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 871/1745] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:44.438765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:44.440298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:44.485337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:44.560174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34070c-14c2-4ba0-9e10-ff69bb6778d3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:44.574990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:44.577654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:44.577961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:44.579930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:44.605763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:44.611480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:44.639047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 875/1746] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:45.348698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:45.350463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:45.365202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:45.369591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d68ce-aae3-4b51-b82f-d7d9ec5a3963 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:45.377748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:45.378577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:45.379207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:45.379480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] PATH_INFO: `/v3/projects/66ce5d5d6c3d45d19ecc15e178eb1adb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:45.381000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ce5d5d6c3d45d19ecc15e178eb1adb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:45.384461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:45.458913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ce5d5d6c3d45d19ecc15e178eb1adb', 'request_id': 'req-6bac4a96-51d5-4414-a565-07d5717e67aa'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.470019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ce5d5d6c3d45d19ecc15e178eb1adb', 'request_id': 'req-6bac4a96-51d5-4414-a565-07d5717e67aa'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.482765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ce5d5d6c3d45d19ecc15e178eb1adb', 'request_id': 'req-6bac4a96-51d5-4414-a565-07d5717e67aa'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.517802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] The token cache is being invalidate because project 66ce5d5d6c3d45d19ecc15e178eb1adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:45.517802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.518394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.522123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 872/1747] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:45 2022] DELETE /identity/v3/projects/66ce5d5d6c3d45d19ecc15e178eb1adb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:45.526218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:45.539838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2151181-9eb9-4534-af0b-f6a822695c13 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:45.692479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:45.693327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:45.698204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:45.698592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] PATH_INFO: `/v3/users/936cf329c85746d1a0edaf8e737667f5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:45.702121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=936cf329c85746d1a0edaf8e737667f5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:45.704705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:45.815864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936cf329c85746d1a0edaf8e737667f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.843615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936cf329c85746d1a0edaf8e737667f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.852298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936cf329c85746d1a0edaf8e737667f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.886819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936cf329c85746d1a0edaf8e737667f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.888492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5f3070bf-ca6d-48d3-8457-61883b76d0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936cf329c85746d1a0edaf8e737667f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:45.937313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 876/1748] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:45 2022] DELETE /identity/v3/users/936cf329c85746d1a0edaf8e737667f5 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:46.174355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:46.176193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:46.190855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac4a96-51d5-4414-a565-07d5717e67aa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:46.286427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:46.287611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:46.288103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:46.288476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] PATH_INFO: `/v3/users/fb03fd5a46e54d06baa81e38679e3134` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:46.290127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb03fd5a46e54d06baa81e38679e3134)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:46.293414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:46.360771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb03fd5a46e54d06baa81e38679e3134'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.373120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb03fd5a46e54d06baa81e38679e3134'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.374245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb03fd5a46e54d06baa81e38679e3134'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.396365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb03fd5a46e54d06baa81e38679e3134'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.404690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb03fd5a46e54d06baa81e38679e3134'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.411062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 873/1749] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:46 2022] DELETE /identity/v3/users/fb03fd5a46e54d06baa81e38679e3134 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:46.416019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:46.416297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:46.416569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:46.440823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:46.442235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:46.498397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:46.520411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:46.523316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:46.529956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 877/1750] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:46.546310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1d4de-3c92-465b-b779-47936b39f01a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:46.551754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:46.552524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:46.552926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:46.553317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] PATH_INFO: `/v3/projects/96d4213d17c944cdb6a7f17d65adecd2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:46.554616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d4213d17c944cdb6a7f17d65adecd2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:46.557664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:46.559429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:46.561290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:46.564091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:46.662249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d4213d17c944cdb6a7f17d65adecd2', 'request_id': 'req-a3130200-6336-4471-b0b9-1ea1880275d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.669756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d4213d17c944cdb6a7f17d65adecd2', 'request_id': 'req-a3130200-6336-4471-b0b9-1ea1880275d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.686950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d4213d17c944cdb6a7f17d65adecd2', 'request_id': 'req-a3130200-6336-4471-b0b9-1ea1880275d2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.717212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] The token cache is being invalidate because project 96d4213d17c944cdb6a7f17d65adecd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:46.717541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.718190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a3130200-6336-4471-b0b9-1ea1880275d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:46.728939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 874/1751] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:46 2022] DELETE /identity/v3/projects/96d4213d17c944cdb6a7f17d65adecd2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:46.731767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb770-3d44-4590-9ebb-ee923aee5374 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:46.732161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb770-3d44-4590-9ebb-ee923aee5374 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:46.732506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb770-3d44-4590-9ebb-ee923aee5374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:46.776229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3b3eb770-3d44-4590-9ebb-ee923aee5374 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:46.777500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b3eb770-3d44-4590-9ebb-ee923aee5374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:46.777747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e53c5-fbf1-4f94-835a-cf8cac290798 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:46.946573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:46.947241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:46.947519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:46.947788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:46.952676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:46.961457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 875/1752] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:46.975950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:46.976300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:46.979283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.025999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.027904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.045713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 878/1753] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:47.047911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:47.049121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:47.064464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.070180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 876/1754] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:47.076582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef1444-0225-45dc-99de-80292871de69 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.083919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.084753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.085355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.085642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.086957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:47.086957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.089789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.112257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.119855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 879/1755] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:47.135660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc2829-d740-41f9-868b-2b2c851db3b6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.153049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.154017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.154429 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.154831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.156333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.158609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.216460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 877/1756] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:47 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:47.219230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.236160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82dfc5-8fa3-4e49-aa82-1e2f90994cfa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.346012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.347105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.347530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.347937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.349096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.351826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.404071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 880/1757] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:47 2022] POST /identity/v3/users => generated 304 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:47.405752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.420923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e774ec-52e8-4b5c-a577-df9a9b393059 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.426745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.427624 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.428038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.428518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.429420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.431130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.443931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 878/1758] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:47.448420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.459914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855b433-f9a9-44b9-9470-6dd5b9a0fabc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.465885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd58f-7e52-4a5d-be05-d23b71d08bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.466988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd58f-7e52-4a5d-be05-d23b71d08bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.467331 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd58f-7e52-4a5d-be05-d23b71d08bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.467926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd58f-7e52-4a5d-be05-d23b71d08bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.468716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd58f-7e52-4a5d-be05-d23b71d08bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.470668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd58f-7e52-4a5d-be05-d23b71d08bb0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.496619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 881/1759] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:47.501288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.521882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168568-0c20-49f7-97b1-b2378882fbf0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.548851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.549600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.549992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.550342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] PATH_INFO: `/v3/projects/1a0ff6e1b0dc4d22937ac84c66878098/users/477749501d77444a83bc30356909aef5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.594219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0ff6e1b0dc4d22937ac84c66878098, user_id=477749501d77444a83bc30356909aef5, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.602817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:47.632742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 879/1760] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:47 2022] PUT /identity/v3/projects/1a0ff6e1b0dc4d22937ac84c66878098/users/477749501d77444a83bc30356909aef5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:47.655423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dad86-1a32-466b-8f18-6435d40baad2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.655878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dad86-1a32-466b-8f18-6435d40baad2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.656119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dad86-1a32-466b-8f18-6435d40baad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.673279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:47.711353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c72dad86-1a32-466b-8f18-6435d40baad2 None None] MFA Rules not processed for user `477749501d77444a83bc30356909aef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:47.713438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c72dad86-1a32-466b-8f18-6435d40baad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:47.853913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 882/1761] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:47.858432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:47.888850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8c731-9838-4dfd-bb51-cd19ed35146c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.889644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8c731-9838-4dfd-bb51-cd19ed35146c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.890292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8c731-9838-4dfd-bb51-cd19ed35146c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.892313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 883/1762] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:37:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:37:47.908800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:47.925061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.925412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.925976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.960167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:47.961496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:47.965937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188203-9c50-437d-972f-9c6b36540dda admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:47.972391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:47.974373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:47.975514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:47.976577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:47.977618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:47.982232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.062753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 880/1763] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:47 2022] POST /identity/v3/projects => generated 407 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:48.065969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.068677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.083862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 884/1764] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:37:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:48.085050 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.117150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddf4de-e757-44ec-aed4-a3535149c0ef None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.223416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beda50-b32e-42f6-82fc-4483edb0e703 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.236744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.237471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.238021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.238291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] PATH_INFO: `/v3/projects/4910d38f8e3141c6895f9f9dc3e10834` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.239494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4910d38f8e3141c6895f9f9dc3e10834)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.241746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.265623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.266258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.266534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.266803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.267472 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.274522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.317852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4910d38f8e3141c6895f9f9dc3e10834', 'request_id': 'req-18d1d6b7-1c12-4053-8867-d06f686881e4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.321456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 885/1765] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:48 2022] POST /identity/v3/users => generated 310 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:48.322437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.328237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.328427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4910d38f8e3141c6895f9f9dc3e10834', 'request_id': 'req-18d1d6b7-1c12-4053-8867-d06f686881e4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.335783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4910d38f8e3141c6895f9f9dc3e10834', 'request_id': 'req-18d1d6b7-1c12-4053-8867-d06f686881e4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.356697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] The token cache is being invalidate because project 4910d38f8e3141c6895f9f9dc3e10834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:48.358090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.358090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.360383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 881/1766] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:48 2022] DELETE /identity/v3/projects/4910d38f8e3141c6895f9f9dc3e10834 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:48.361355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.364723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.367637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:48.436604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:48.467817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d6b7-1c12-4053-8867-d06f686881e4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.471107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edad60-b120-4283-b42d-b6741c9ff475 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.473250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4564c79-2793-4407-bfd7-4b32b03e27a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.474092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4564c79-2793-4407-bfd7-4b32b03e27a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.474436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4564c79-2793-4407-bfd7-4b32b03e27a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.474795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4564c79-2793-4407-bfd7-4b32b03e27a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.475570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4564c79-2793-4407-bfd7-4b32b03e27a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.478050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4564c79-2793-4407-bfd7-4b32b03e27a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.492563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 882/1767] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:48.494052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.494404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.494783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.518925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:48.520324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.531796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 883/1768] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:48.539178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.556364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d26d0-eb6d-43bf-96be-7eb849ea351b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.563243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.564037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.564451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.564774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] PATH_INFO: `/v3/projects/2cf4842b048840729267ceeaf383e9bb/users/7c7475f08b6543ee930c7891720ec6a2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.574270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.574916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.575190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.575588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.578141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.588448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf4842b048840729267ceeaf383e9bb, user_id=7c7475f08b6543ee930c7891720ec6a2, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.593053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.593301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.594818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0b1e-4603-4511-93b3-4c3df18a65d9 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.609292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 886/1769] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:48.610943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.611406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.611909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.628173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 884/1770] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:48 2022] PUT /identity/v3/projects/2cf4842b048840729267ceeaf383e9bb/users/7c7475f08b6543ee930c7891720ec6a2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:48.632676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.651014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:48.652258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.709956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ee801-3ad3-4ab4-a37c-7a9c3010ca7e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.717458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.718476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.718938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.719761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.720649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.722980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.737879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 885/1771] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:37:48.745381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.749134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.768036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420337c-6c68-4b64-a3f8-7a87b086ca40 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.778941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.780592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.781472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.782365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] PATH_INFO: `/v3/users/be18e2793c0048c8abee48727a35bbae` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.785405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be18e2793c0048c8abee48727a35bbae)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.789559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.809894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 887/1772] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:48.812205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.835500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863257d-f044-40af-90ed-16fac1bf3816 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:48.846229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:48.846919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.847204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.847473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] PATH_INFO: `/v3/projects/2cf4842b048840729267ceeaf383e9bb/users/7c7475f08b6543ee930c7891720ec6a2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.849694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf4842b048840729267ceeaf383e9bb, user_id=7c7475f08b6543ee930c7891720ec6a2, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:48.852756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:48.872517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 888/1773] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:48 2022] PUT /identity/v3/projects/2cf4842b048840729267ceeaf383e9bb/users/7c7475f08b6543ee930c7891720ec6a2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:48.874706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:48.891748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be18e2793c0048c8abee48727a35bbae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.904826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be18e2793c0048c8abee48727a35bbae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.905942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be18e2793c0048c8abee48727a35bbae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.928803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be18e2793c0048c8abee48727a35bbae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.933048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:48.937881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48500de9-83d2-4675-9f9c-84efe7e490a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be18e2793c0048c8abee48727a35bbae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:48.946319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 886/1774] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:48 2022] DELETE /identity/v3/users/be18e2793c0048c8abee48727a35bbae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:48.947925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452ef42-e19d-4bc9-9092-15cbbbdc49ca None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:48.948208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452ef42-e19d-4bc9-9092-15cbbbdc49ca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:48.948477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452ef42-e19d-4bc9-9092-15cbbbdc49ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:48.982698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7452ef42-e19d-4bc9-9092-15cbbbdc49ca None None] MFA Rules not processed for user `7c7475f08b6543ee930c7891720ec6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:48.984299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7452ef42-e19d-4bc9-9092-15cbbbdc49ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:48.989534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:48.995547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7062b62-0fd3-46bb-ad5f-4144ea228dac admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.081314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.082069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.082410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.082740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.084063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:49.084427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.087163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.089702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 887/1775] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:48 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:49.091485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.091907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.092490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.109556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 889/1776] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:37:49.110852 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.116709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.124006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:49.125165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.141160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 888/1777] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:49.142125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.143646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.157001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261eadf-825a-4650-a039-c6871fe9cf97 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.163798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.164845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.165192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.165535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.166609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.168638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.210422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 889/1778] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:49 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:49.214229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.217690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6d829-5c55-47fa-8875-e9304f4502bf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.218316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.297695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.298392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.298856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.299193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.330622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482d1d-4783-4423-8016-d7e6d41110ab admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.336971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c9163-cdbe-4cc2-82cf-c6d287371e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.338245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c9163-cdbe-4cc2-82cf-c6d287371e9d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.338592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c9163-cdbe-4cc2-82cf-c6d287371e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.338945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c9163-cdbe-4cc2-82cf-c6d287371e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.351676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:49.397407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.399659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.417439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 890/1779] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:49 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:49.419247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.468697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c9163-cdbe-4cc2-82cf-c6d287371e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.471792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c9163-cdbe-4cc2-82cf-c6d287371e9d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.499626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 890/1780] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:49.573654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56626e-e3de-474f-9abd-ba437bd1d344 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.575465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56626e-e3de-474f-9abd-ba437bd1d344 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.576543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56626e-e3de-474f-9abd-ba437bd1d344 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.579201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 891/1781] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:37:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:37:49.591192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.591715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.592123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.614224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaa8ed-2b9a-490b-a667-8e12e091aa7e service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.625964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.626616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.626886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.627150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.627823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.635488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.665768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:49.669015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.683466 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 891/1782] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:49 2022] POST /identity/v3/users => generated 287 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:49.687460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.713703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:49.714543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a7420-f724-4a2a-9468-7a39d66d2680 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.729476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.730354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.730702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.731152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.735063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.743414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.757428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 892/1783] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:49.758496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.773266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c599-c4ad-4d64-b878-46b886739681 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.778219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.779093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.779458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.779887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.795145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.798388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 892/1784] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:37:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:49.799330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.815956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42b14f-7320-4ad6-b261-effddb3d1859 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.826653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.827314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.829312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.829664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.830322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.833986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.856094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 893/1785] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:49.858497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.861954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:49.877643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.879729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.884254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f991-bf99-4fc5-b1d2-da21cbeeed5e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.894025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205d88d-4806-4d17-b350-69183e98729c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.894760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d88d-4806-4d17-b350-69183e98729c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.895102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d88d-4806-4d17-b350-69183e98729c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.895585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d88d-4806-4d17-b350-69183e98729c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:49.903377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 893/1786] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:49.905495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:49.908531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205d88d-4806-4d17-b350-69183e98729c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:49.910170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205d88d-4806-4d17-b350-69183e98729c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:49.931369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfe66b-5ae0-4c63-ad8a-5ba6da4166cb service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:49.938165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 894/1787] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:49.944513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:49.945418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:49.946526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:49.948742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] PATH_INFO: `/v3/projects/6854add59af44a8ebb1fdd7078c7ea9b/users/62131fe3ea09461ab6fc400c2787dc6d/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.002613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6854add59af44a8ebb1fdd7078c7ea9b, user_id=62131fe3ea09461ab6fc400c2787dc6d, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.021962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.065880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 894/1788] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:49 2022] PUT /identity/v3/projects/6854add59af44a8ebb1fdd7078c7ea9b/users/62131fe3ea09461ab6fc400c2787dc6d/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:50.079174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.079453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.079727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.121831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] MFA Rules not processed for user `62131fe3ea09461ab6fc400c2787dc6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:50.124281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:50.274103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 895/1789] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:50.278952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:50.359173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:50.428971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68d87-1332-4f4d-a86c-bb6c06b68823 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:50.433890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:50.434735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.435223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.435571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.437035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.439663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.480901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 895/1790] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:50 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:50.486242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:50.501574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46f5cf-9b28-4ab5-b642-75cf099e3770 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:50.651155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:50.652029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.652454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.653052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.653929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.656095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.686134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 896/1791] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:50 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:50.691665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:50.703528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b3864-b4b4-4646-afdc-65b78b7605d0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:50.713085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:50.713085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.713395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.713395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.714449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.716570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.734458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 896/1792] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:50.735694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:50.749131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444957a3-e18e-42c8-b207-c5b9f1f20f2c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:50.755761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:50.757214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.757583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.758362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] PATH_INFO: `/v3/projects/2c67625559fb4f98b5adb95fb35c9eef/users/0c2730df34fd4a3f916993a6310c854b/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.765081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:50.790017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c67625559fb4f98b5adb95fb35c9eef, user_id=0c2730df34fd4a3f916993a6310c854b, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.794896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.819302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 897/1793] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:50 2022] PUT /identity/v3/projects/2c67625559fb4f98b5adb95fb35c9eef/users/0c2730df34fd4a3f916993a6310c854b/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:50.823383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:50.833655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e418a3-adaa-4275-abb9-18da7dac1cb4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:50.912623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd72ece-9139-44e9-9a99-e5a0867d40ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:50.913378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd72ece-9139-44e9-9a99-e5a0867d40ca admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.913728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd72ece-9139-44e9-9a99-e5a0867d40ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.913993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd72ece-9139-44e9-9a99-e5a0867d40ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.914991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd72ece-9139-44e9-9a99-e5a0867d40ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.916371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd72ece-9139-44e9-9a99-e5a0867d40ca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.928801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 897/1794] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:50.932040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:50.943315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2a9d0-b937-4574-9273-a24aad23a175 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:50.948291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:50.948704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.949011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.949325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] PATH_INFO: `/v3/projects/2c67625559fb4f98b5adb95fb35c9eef/users/0c2730df34fd4a3f916993a6310c854b/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:50.951724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c67625559fb4f98b5adb95fb35c9eef, user_id=0c2730df34fd4a3f916993a6310c854b, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:50.956173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:50.981286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 898/1795] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:50 2022] PUT /identity/v3/projects/2c67625559fb4f98b5adb95fb35c9eef/users/0c2730df34fd4a3f916993a6310c854b/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:50.983246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:50.983530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:50.983798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:51.023075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] MFA Rules not processed for user `0c2730df34fd4a3f916993a6310c854b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:51.024799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:51.101051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 898/1796] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:51.121063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:51.211863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa80a8a-af3b-4c20-8e17-c1081b02fe7c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:51.216482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:51.228118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:51.228410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:51.228693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:51.249173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:51.250645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:51.286537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 899/1797] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:52.057194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:52.101974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:52.141605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:52.143650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:52.153324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5dd4d-3c09-4224-b11e-38a004690f8f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:52.158239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b579fd-c2c1-4480-84a6-f5d4d07c3002 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:52.158476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:52.159134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:52.159412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:52.159682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] PATH_INFO: `/v3/users/62131fe3ea09461ab6fc400c2787dc6d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:52.161066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62131fe3ea09461ab6fc400c2787dc6d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:52.163436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:52.165399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:52.166490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:52.167154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:52.167434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] PATH_INFO: `/v3/projects/e281870df3cc401d9876370c8c0b3cc5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:52.168960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e281870df3cc401d9876370c8c0b3cc5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:52.172298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:52.224158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62131fe3ea09461ab6fc400c2787dc6d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.236369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62131fe3ea09461ab6fc400c2787dc6d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.238650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62131fe3ea09461ab6fc400c2787dc6d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.240278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e281870df3cc401d9876370c8c0b3cc5', 'request_id': 'req-e7eead27-3ac4-4281-81df-f21353787014'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.246761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e281870df3cc401d9876370c8c0b3cc5', 'request_id': 'req-e7eead27-3ac4-4281-81df-f21353787014'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.249700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62131fe3ea09461ab6fc400c2787dc6d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.250680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a05bf8a3-6bb5-435f-92d5-e0add83dbda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62131fe3ea09461ab6fc400c2787dc6d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.255168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e281870df3cc401d9876370c8c0b3cc5', 'request_id': 'req-e7eead27-3ac4-4281-81df-f21353787014'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.270788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] The token cache is being invalidate because project e281870df3cc401d9876370c8c0b3cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:52.271073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.271737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.274126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 900/1798] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:52 2022] DELETE /identity/v3/projects/e281870df3cc401d9876370c8c0b3cc5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:52.275148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 899/1799] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:52 2022] DELETE /identity/v3/users/62131fe3ea09461ab6fc400c2787dc6d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:52.278252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:52.280066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:52.281884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:52.282434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:52.283202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:52.308944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:52.310529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:52.368381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:52.378633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eead27-3ac4-4281-81df-f21353787014 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:52.394783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 900/1800] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:52.405391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:52.406825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:52.452063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:52.453060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:52.453334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:52.453599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] PATH_INFO: `/v3/users/0c2be860a25c471aab2f0b15c84cdc78` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:52.456539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2be860a25c471aab2f0b15c84cdc78)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:52.458644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:52.467396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:52.483181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e73d-1fa9-4173-b099-c1e838c77de5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:52.493332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:52.494183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:52.494561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:52.494934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:52.506651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:52.508654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:52.528664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 901/1801] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:52.546441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2be860a25c471aab2f0b15c84cdc78'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.556653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2be860a25c471aab2f0b15c84cdc78'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.557560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2be860a25c471aab2f0b15c84cdc78'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.573776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2be860a25c471aab2f0b15c84cdc78'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.581443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2be860a25c471aab2f0b15c84cdc78'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:52.588135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 901/1802] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:52 2022] DELETE /identity/v3/users/0c2be860a25c471aab2f0b15c84cdc78 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:53.263152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:53.264740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:53.284980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9db5-5089-401d-b48e-3df6c2f469d6 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:53.321383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:53.324074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:53.421145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:53.422499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.423006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.423504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] PATH_INFO: `/v3/projects/6854add59af44a8ebb1fdd7078c7ea9b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.425736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6854add59af44a8ebb1fdd7078c7ea9b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:53.428577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:53.457781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f6407-5fdc-456b-a911-cd77291c7af1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:53.465003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:53.465681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.466109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.466629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] PATH_INFO: `/v3/projects/ef1ebc0a8e124ccebcbcd005308c875c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.468134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1ebc0a8e124ccebcbcd005308c875c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:53.471598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:53.489141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6854add59af44a8ebb1fdd7078c7ea9b', 'request_id': 'req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.496414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6854add59af44a8ebb1fdd7078c7ea9b', 'request_id': 'req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.503697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6854add59af44a8ebb1fdd7078c7ea9b', 'request_id': 'req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.522079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:53.524807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] The token cache is being invalidate because project 6854add59af44a8ebb1fdd7078c7ea9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:53.525386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.526296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.528910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 902/1803] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:53 2022] DELETE /identity/v3/projects/6854add59af44a8ebb1fdd7078c7ea9b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:53.532532 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:53.534212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:53.547680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1ebc0a8e124ccebcbcd005308c875c', 'request_id': 'req-363c8271-9253-44a7-bf74-105ccffce054'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.555176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1ebc0a8e124ccebcbcd005308c875c', 'request_id': 'req-363c8271-9253-44a7-bf74-105ccffce054'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.563736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1ebc0a8e124ccebcbcd005308c875c', 'request_id': 'req-363c8271-9253-44a7-bf74-105ccffce054'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.583303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] The token cache is being invalidate because project ef1ebc0a8e124ccebcbcd005308c875c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:53.583654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.584426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-363c8271-9253-44a7-bf74-105ccffce054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.586143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 902/1804] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:53 2022] DELETE /identity/v3/projects/ef1ebc0a8e124ccebcbcd005308c875c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:53.591110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbacee-4af7-4d0c-adb8-60de69b1fe30 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.591689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbacee-4af7-4d0c-adb8-60de69b1fe30 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.592205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbacee-4af7-4d0c-adb8-60de69b1fe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.634107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a7dbacee-4af7-4d0c-adb8-60de69b1fe30 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:53.635573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7dbacee-4af7-4d0c-adb8-60de69b1fe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:53.642025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d318bb1-7a63-4cf9-acbc-3f98a01bcea8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:53.666623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:53.716230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 903/1805] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:53.718682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.719088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.719725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.733166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:53.733861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.734146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.734468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] PATH_INFO: `/v3/users/0c2730df34fd4a3f916993a6310c854b` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.737027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2730df34fd4a3f916993a6310c854b)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:53.739647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:53.752048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:53.754367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:53.766592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 904/1806] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:53.771383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:53.782824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c866f5-b496-46b2-800f-484fe6236ffd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:53.791179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:53.792174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.792627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.793112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.794013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:53.794558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:53.796486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:53.817008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 905/1807] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:37:53.822074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:53.837385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6c061-a038-4ddd-8288-e9bfe4ce8a98 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:53.848538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:53.849380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:53.849827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:53.851511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:53.851988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:53.854004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:53.855859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2730df34fd4a3f916993a6310c854b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.886667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2730df34fd4a3f916993a6310c854b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.888389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2730df34fd4a3f916993a6310c854b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.898944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 906/1808] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:53 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:53.903534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:53.915918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2730df34fd4a3f916993a6310c854b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.916889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2730df34fd4a3f916993a6310c854b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:53.948932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 903/1809] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:53 2022] DELETE /identity/v3/users/0c2730df34fd4a3f916993a6310c854b => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:53.994260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb502-308d-4411-aaa7-980fbce25b0c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.013444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:54.063785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.064528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.064946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.065333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.066388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.068431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.096159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 907/1810] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:53 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:54.097956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.099705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:54.116162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad4780-66e8-41a7-9fdd-9d64036e830d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.121341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.121971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.122252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.122560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.123260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.124917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.135529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 904/1811] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:54.139668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.149849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d514-2078-4184-8d63-0ad4258eb169 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.154395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.155013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.155285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.155550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.156238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.157864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.168053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 908/1812] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:54.172438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.182813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd617d-de88-4da7-a542-a741df0e5882 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.187839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f46d0-f94f-432f-9c0c-29968584a1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.188512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f46d0-f94f-432f-9c0c-29968584a1e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.188815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f46d0-f94f-432f-9c0c-29968584a1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.189087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f46d0-f94f-432f-9c0c-29968584a1e1 admin admin] PATH_INFO: `/v3/projects/70f81077984a4904a4c20cf5cce1bdb4/users/f3b1f4f1d0a141b7a7494f9323ed0074/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.204533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f46d0-f94f-432f-9c0c-29968584a1e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f81077984a4904a4c20cf5cce1bdb4, user_id=f3b1f4f1d0a141b7a7494f9323ed0074, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.207450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f46d0-f94f-432f-9c0c-29968584a1e1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.221148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 905/1813] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:54 2022] PUT /identity/v3/projects/70f81077984a4904a4c20cf5cce1bdb4/users/f3b1f4f1d0a141b7a7494f9323ed0074/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:54.231163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.233144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:54.233809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.234094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.234366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.248334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47139c38-a3e7-4af2-b434-5337d2dc6786 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.270747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] MFA Rules not processed for user `f3b1f4f1d0a141b7a7494f9323ed0074`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:54.272039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:54.309831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.310518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.310839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.311109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] PATH_INFO: `/v3/projects/2c67625559fb4f98b5adb95fb35c9eef` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.312271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c67625559fb4f98b5adb95fb35c9eef)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.314281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.330430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:54.339821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 906/1814] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:54.340663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.353363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f735a-6065-49dc-8f3b-ef4ddb70f549 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.362727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c67625559fb4f98b5adb95fb35c9eef', 'request_id': 'req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:54.368865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c67625559fb4f98b5adb95fb35c9eef', 'request_id': 'req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:54.375556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c67625559fb4f98b5adb95fb35c9eef', 'request_id': 'req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:54.392727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] The token cache is being invalidate because project 2c67625559fb4f98b5adb95fb35c9eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:54.393108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:54.393786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f9f7efbb-e3a6-4fca-9f3d-1cee259ce42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:54.396832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 909/1815] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:54 2022] DELETE /identity/v3/projects/2c67625559fb4f98b5adb95fb35c9eef => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:54.402404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df30d19-0116-429a-90a7-9165f887b9f8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.402863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df30d19-0116-429a-90a7-9165f887b9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.403141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df30d19-0116-429a-90a7-9165f887b9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.420965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.421833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.421956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.422164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.422830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.424712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.425845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4df30d19-0116-429a-90a7-9165f887b9f8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:54.427386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4df30d19-0116-429a-90a7-9165f887b9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:54.437756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 910/1816] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:54.444819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.445178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.445526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.467006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 907/1817] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:54.469903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:54.471225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.473046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:54.473682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:54.495029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:54.555301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373fe24-4451-4c3e-af96-6df24a76594d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.556015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 911/1818] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:54.560742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.565156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.565954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.566232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.566502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.567702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.570484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a73c85-d885-4582-872e-4443ac63a7dd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.570632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.575387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.576147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.576485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.576875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.577622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:54.577988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.579650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.594829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 912/1819] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:37:54.596825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 908/1820] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:54.600591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.600979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.611191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ac627-b0cc-4d06-876e-995ac788f1a1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.612263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dea179-45a7-4817-b687-b594ad4c2483 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.616094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.616885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.617172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.617279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.617576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.617844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.618122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.618304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.618405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.619037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.619884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.621016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.635346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 909/1821] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:54.640110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.659278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 913/1822] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:54.662450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea615-94b7-4a8a-b1bf-04679c5abf7d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.663844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.735026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a1942-14a6-4ed8-be02-056c3b02532c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.737409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.738131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.738364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.738643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] PATH_INFO: `/v3/projects/3c4d29840f65442885b93a4558895ddf/users/7274cc4a0def4ffabfd5c08136f7162e/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.739796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.740422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.740695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.741101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.741756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.743078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:54.743915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.760217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4d29840f65442885b93a4558895ddf, user_id=7274cc4a0def4ffabfd5c08136f7162e, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.763850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.769065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 914/1823] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:54.773972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.784023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 910/1824] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:54 2022] PUT /identity/v3/projects/3c4d29840f65442885b93a4558895ddf/users/7274cc4a0def4ffabfd5c08136f7162e/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:54.787206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c710a-c323-4dad-85c8-5e29b9e5ccf0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.787531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.854851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaad0fb-8b8d-4372-b4f6-11686d2fdb65 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.859698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.860350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.860629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.860943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.861651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.863253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.866889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.867521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.867797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.868063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.868789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.870411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.874240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 911/1825] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:54.878625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.884106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 915/1826] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:54.885457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.890422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2048bbd-71ba-44ba-bb03-46399f886f2f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.895692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.896417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517b839-9df9-4e8c-a9f3-755acfbd37e7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:54.898198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.898473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.899597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] PATH_INFO: `/v3/projects/3c4d29840f65442885b93a4558895ddf/users/7274cc4a0def4ffabfd5c08136f7162e/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.902018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4d29840f65442885b93a4558895ddf, user_id=7274cc4a0def4ffabfd5c08136f7162e, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.902659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb31299-8bf9-42ce-a3a0-1b5a84709eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:54.903397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb31299-8bf9-42ce-a3a0-1b5a84709eee admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.903726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb31299-8bf9-42ce-a3a0-1b5a84709eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.904110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb31299-8bf9-42ce-a3a0-1b5a84709eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.904979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.905272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb31299-8bf9-42ce-a3a0-1b5a84709eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:54.907459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb31299-8bf9-42ce-a3a0-1b5a84709eee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:54.918922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 912/1827] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:54 2022] PUT /identity/v3/projects/3c4d29840f65442885b93a4558895ddf/users/7274cc4a0def4ffabfd5c08136f7162e/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:54.923456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 916/1828] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:37:54.925889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:54.929993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:54.929993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:54.929993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:54.938606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:54.966593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] MFA Rules not processed for user `7274cc4a0def4ffabfd5c08136f7162e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:54.968991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:55.033214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17c459-9b87-412e-8327-0c969c28cbba admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.038097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbd6c6-7ff7-45d2-b195-5f3371716471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.039437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbd6c6-7ff7-45d2-b195-5f3371716471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.039437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbd6c6-7ff7-45d2-b195-5f3371716471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.039437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbd6c6-7ff7-45d2-b195-5f3371716471 admin admin] PATH_INFO: `/v3/projects/ceb80b412d08489ca61dd5d699eb4501/users/cca2ed267659436a90ce286581c6f065/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.084526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 917/1829] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:55.084937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbd6c6-7ff7-45d2-b195-5f3371716471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb80b412d08489ca61dd5d699eb4501, user_id=cca2ed267659436a90ce286581c6f065, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.084937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbd6c6-7ff7-45d2-b195-5f3371716471 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.102658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.104269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:55.113144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 913/1830] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:54 2022] PUT /identity/v3/projects/ceb80b412d08489ca61dd5d699eb4501/users/cca2ed267659436a90ce286581c6f065/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:55.125209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.125797 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.126241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.191359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] MFA Rules not processed for user `cca2ed267659436a90ce286581c6f065`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:55.192618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:55.225632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4dca1-1bc7-4b48-91e7-63b892c853cd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.257039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:55.296807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 914/1831] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:37:55.301135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.323288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.324075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.324414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.325093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.379581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53443eb-d609-4cfc-b920-ab6dbfd94891 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.384875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.385590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.385869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.386139 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.386864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.388925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.403645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:55.407362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.409109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.420357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 918/1832] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:55.421854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 915/1833] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:55 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:55.425754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.437532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d2a7-536e-4be4-9890-a2793c14e8ae service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.508047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.508810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.509137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.509415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.510086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.512005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.544839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 919/1834] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:37:55 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:37:55.547021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.562101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff6e1b-34b1-4087-ba40-61ab7e5f10e5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.569876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.570512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.570788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.571139 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.571820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.573586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.590085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 916/1835] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:37:55.591679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.611295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408342-1aa6-4f8d-a1e8-d78646e83f26 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.620162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.621282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.626981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.626981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] PATH_INFO: `/v3/projects/5335d3b4882446bf96e29f19dd358253/users/74b9c644b50f4b5bbfd12b069805d17a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.645740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5335d3b4882446bf96e29f19dd358253, user_id=74b9c644b50f4b5bbfd12b069805d17a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.648863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.668106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 920/1836] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:55 2022] PUT /identity/v3/projects/5335d3b4882446bf96e29f19dd358253/users/74b9c644b50f4b5bbfd12b069805d17a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:55.671935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.682953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573a5e-e9b1-4ced-91df-9a3c1f955b10 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.793334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ed895-2709-4834-817b-af1991a5838f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.794225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ed895-2709-4834-817b-af1991a5838f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.794693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ed895-2709-4834-817b-af1991a5838f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.795107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ed895-2709-4834-817b-af1991a5838f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.796132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ed895-2709-4834-817b-af1991a5838f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.798930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ed895-2709-4834-817b-af1991a5838f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.818209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 917/1837] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:37:55.819348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.832678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba696f-7ca9-4062-871c-bfe7b6fadb29 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.841786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.842487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.842764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.843038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] PATH_INFO: `/v3/projects/5335d3b4882446bf96e29f19dd358253/users/74b9c644b50f4b5bbfd12b069805d17a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.845395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5335d3b4882446bf96e29f19dd358253, user_id=74b9c644b50f4b5bbfd12b069805d17a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.855660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.872840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 921/1838] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:37:55 2022] PUT /identity/v3/projects/5335d3b4882446bf96e29f19dd358253/users/74b9c644b50f4b5bbfd12b069805d17a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:55.883542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.883542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.883542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.884588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:55.925190 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] MFA Rules not processed for user `74b9c644b50f4b5bbfd12b069805d17a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:55.926412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:55.966503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8df276-32f6-4000-923a-6e3f4db08182 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:55.972742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7274cc4a0def4ffabfd5c08136f7162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4d29840f65442885b93a4558895ddf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:55.973891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:55.974659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:55.975232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:55.976821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:55.977384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:55.979647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:55.996647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 922/1839] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:37:56.001358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:56.012914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:56.013638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542b0bb-79ae-4812-a9ef-debd9ec0dc42 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:56.027184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7274cc4a0def4ffabfd5c08136f7162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4d29840f65442885b93a4558895ddf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:56.027826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:56.028106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:56.028379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:56.029149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:56.031085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:56.046243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 918/1840] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:56.058288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:56.075960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 923/1841] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:37:56 2022] POST /identity/v3/projects => generated 354 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:37:56.083026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:56.111295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:56.125908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b60ebc-a973-4834-bf7b-4982d8b80234 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:56.163520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42265cca-9d05-4cb8-82ab-fb44d8482ee5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:56.168599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7274cc4a0def4ffabfd5c08136f7162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4d29840f65442885b93a4558895ddf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:56.169340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:56.169626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:56.169901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:56.171123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:37:56.171123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:56.172573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:56.184086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:56.184706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:56.188411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:56.188525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 924/1842] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:37:56.188631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:56.226360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:56.237601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483876ff-798e-44d0-ba16-c84eb2ee96a5 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:56.242516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7274cc4a0def4ffabfd5c08136f7162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4d29840f65442885b93a4558895ddf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:56.243191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:56.243468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:56.243742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] PATH_INFO: `/v3/projects/82fe62f5e0bf4bc6aec66ac912602bfa` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:56.245104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorizing `identity:delete_project(project_id=82fe62f5e0bf4bc6aec66ac912602bfa)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:56.248015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:56.275750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:56.277730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:56.292197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 919/1843] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:56 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:37:56.314508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82fe62f5e0bf4bc6aec66ac912602bfa', 'request_id': 'req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.325644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82fe62f5e0bf4bc6aec66ac912602bfa', 'request_id': 'req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.334814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82fe62f5e0bf4bc6aec66ac912602bfa', 'request_id': 'req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.353515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] The token cache is being invalidate because project 82fe62f5e0bf4bc6aec66ac912602bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:56.353907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.354641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a0bbb9d3-5195-4dc9-ad8f-506ebd1d7c91 tempest-VolumeQuotaClassesTest-14230551 tempest-VolumeQuotaClassesTest-14230551-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.360667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 925/1844] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:56 2022] DELETE /identity/v3/projects/82fe62f5e0bf4bc6aec66ac912602bfa => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:37:56.682830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:56.743039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:56.767282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfdc55-4e68-49c4-8632-dbc0598622fa service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:56.857714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:56.858861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:56.859343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:56.859941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] PATH_INFO: `/v3/users/f3b1f4f1d0a141b7a7494f9323ed0074` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:56.861535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b1f4f1d0a141b7a7494f9323ed0074)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:56.864099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:56.972943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b1f4f1d0a141b7a7494f9323ed0074'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.993859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b1f4f1d0a141b7a7494f9323ed0074'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:56.995000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b1f4f1d0a141b7a7494f9323ed0074'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:57.039923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b1f4f1d0a141b7a7494f9323ed0074'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:57.042390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b1f4f1d0a141b7a7494f9323ed0074'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:57.090093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 920/1845] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:56 2022] DELETE /identity/v3/users/f3b1f4f1d0a141b7a7494f9323ed0074 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:57.094776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:57.095278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:57.095465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:57.137603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:57.138965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:57.217957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 926/1846] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:57.232395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:57.235726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:57.300941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9c7b9-8ab3-40b1-a6ab-fb55a2016f8e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:57.304786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:57.305422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:57.305708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:57.305960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:57.316189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:57.318062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:57.328682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 921/1847] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:58.171648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:58.173831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.188845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:58.190955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.232037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca9ac9-84f1-48ce-b5bd-1459299fbc6d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:58.240013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342eb83-74e4-4b5f-b32b-af28cdc8380f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:58.245543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:58.246184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.246452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.246720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] PATH_INFO: `/v3/users/cca2ed267659436a90ce286581c6f065` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.248188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca2ed267659436a90ce286581c6f065)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:58.250728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:58.278222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:58.278971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.279249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.286685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] PATH_INFO: `/v3/users/477749501d77444a83bc30356909aef5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.303172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] RBAC: Authorizing `identity:delete_user(user_id=477749501d77444a83bc30356909aef5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:58.310801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:58.324548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:58.354512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:58.382565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca2ed267659436a90ce286581c6f065'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.401151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477749501d77444a83bc30356909aef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.401837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca2ed267659436a90ce286581c6f065'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.402549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca2ed267659436a90ce286581c6f065'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.428343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477749501d77444a83bc30356909aef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.429491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477749501d77444a83bc30356909aef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.431412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca2ed267659436a90ce286581c6f065'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.443597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca2ed267659436a90ce286581c6f065'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.448897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477749501d77444a83bc30356909aef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.450898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f21cd8cf-9f96-45d3-8669-e1f5124a462e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477749501d77444a83bc30356909aef5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.451630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 927/1848] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:58 2022] DELETE /identity/v3/users/cca2ed267659436a90ce286581c6f065 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:58.456494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:58.458437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.476920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 922/1849] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:58 2022] DELETE /identity/v3/users/477749501d77444a83bc30356909aef5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:58.478037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72be60e-00a6-4355-b10c-50d54a51524f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.478323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72be60e-00a6-4355-b10c-50d54a51524f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.478596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72be60e-00a6-4355-b10c-50d54a51524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.500948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f72be60e-00a6-4355-b10c-50d54a51524f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:58.502293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f72be60e-00a6-4355-b10c-50d54a51524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.536597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772e814-368c-423c-8f8d-5228f8807116 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:58.569796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 923/1850] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:58.571830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.572114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.572651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.596557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:37:58.598581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.613058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:58.613968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.614711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.615389 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] PATH_INFO: `/v3/projects/70f81077984a4904a4c20cf5cce1bdb4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.617707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f81077984a4904a4c20cf5cce1bdb4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:58.621948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:58.630437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 924/1851] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:37:58.631352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:58.632878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.696640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f81077984a4904a4c20cf5cce1bdb4', 'request_id': 'req-8d820afd-16ac-43f8-9c13-1223f99735c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.704877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f81077984a4904a4c20cf5cce1bdb4', 'request_id': 'req-8d820afd-16ac-43f8-9c13-1223f99735c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.708319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:58.720814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f81077984a4904a4c20cf5cce1bdb4', 'request_id': 'req-8d820afd-16ac-43f8-9c13-1223f99735c7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.757935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] The token cache is being invalidate because project 70f81077984a4904a4c20cf5cce1bdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:58.758342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.759060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:58.761651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 928/1852] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:58 2022] DELETE /identity/v3/projects/70f81077984a4904a4c20cf5cce1bdb4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:58.763177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:58.765484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.796342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b56316-b479-4a53-b46c-93b20026a6b9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:58.852366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d820afd-16ac-43f8-9c13-1223f99735c7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:58.854603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:58.856748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:58.857494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.857772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.858042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.862386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.889390 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:58.890740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:58.891021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:58.891295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:58.895196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.953372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:58.954924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:58.966394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 929/1853] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:58.967271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:58.968866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:58.986265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d6505-475f-4281-869c-cf7ab7ad3b68 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:58.998565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:58.999725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:59.005154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:59.006681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] PATH_INFO: `/v3/users/7274cc4a0def4ffabfd5c08136f7162e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:59.010115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:59.011946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:59.013733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7274cc4a0def4ffabfd5c08136f7162e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:59.015713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:59.029826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 925/1854] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:37:59.118211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7274cc4a0def4ffabfd5c08136f7162e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.132555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7274cc4a0def4ffabfd5c08136f7162e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.133933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7274cc4a0def4ffabfd5c08136f7162e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.157729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7274cc4a0def4ffabfd5c08136f7162e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.172058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7274cc4a0def4ffabfd5c08136f7162e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.180435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 930/1855] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:58 2022] DELETE /identity/v3/users/7274cc4a0def4ffabfd5c08136f7162e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:59.431412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:59.432877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:59.456435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de2a83-5836-4bb6-8b2c-290f73ba2e6b service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:59.550369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:59.551981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:59.551981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:59.552256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] PATH_INFO: `/v3/projects/1a0ff6e1b0dc4d22937ac84c66878098` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:59.554731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0ff6e1b0dc4d22937ac84c66878098)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:59.556325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:59.618153 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0ff6e1b0dc4d22937ac84c66878098', 'request_id': 'req-7b169001-a8f2-4f16-805f-d5204dd0b498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.629352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0ff6e1b0dc4d22937ac84c66878098', 'request_id': 'req-7b169001-a8f2-4f16-805f-d5204dd0b498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.637161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0ff6e1b0dc4d22937ac84c66878098', 'request_id': 'req-7b169001-a8f2-4f16-805f-d5204dd0b498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.655961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] The token cache is being invalidate because project 1a0ff6e1b0dc4d22937ac84c66878098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:37:59.656358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.657609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.662988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 926/1856] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:37:59 2022] DELETE /identity/v3/projects/1a0ff6e1b0dc4d22937ac84c66878098 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:37:59.663713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:37:59.665159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:37:59.698338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f3bc2-0e09-4128-a779-bef675b14af2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:37:59.707141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:37:59.774724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:37:59.776143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:37:59.776143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:37:59.776143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] PATH_INFO: `/v3/users/7c7475f08b6543ee930c7891720ec6a2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:37:59.777993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c7475f08b6543ee930c7891720ec6a2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:37:59.780267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:37:59.837226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c7475f08b6543ee930c7891720ec6a2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.848483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c7475f08b6543ee930c7891720ec6a2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.849610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c7475f08b6543ee930c7891720ec6a2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.877881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c7475f08b6543ee930c7891720ec6a2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.889489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c7475f08b6543ee930c7891720ec6a2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:37:59.905110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 931/1857] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:37:59 2022] DELETE /identity/v3/users/7c7475f08b6543ee930c7891720ec6a2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:00.286953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:00.288943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:00.294346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:00.295920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:00.315329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169001-a8f2-4f16-805f-d5204dd0b498 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:00.348210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:00.416232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c03d2-44b7-44ad-82fb-670ff4e3a37e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:00.431030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:00.433772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.434305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.434671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] PATH_INFO: `/v3/projects/ceb80b412d08489ca61dd5d699eb4501` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.436972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb80b412d08489ca61dd5d699eb4501)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:00.439283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:00.446932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:00.448151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.448626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.449321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] PATH_INFO: `/v3/projects/3c4d29840f65442885b93a4558895ddf` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.450788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4d29840f65442885b93a4558895ddf)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:00.464829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:00.490817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:00.515019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb80b412d08489ca61dd5d699eb4501', 'request_id': 'req-55297c54-52c9-4096-8d40-aca1ff39ecb6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.524449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb80b412d08489ca61dd5d699eb4501', 'request_id': 'req-55297c54-52c9-4096-8d40-aca1ff39ecb6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.532171 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb80b412d08489ca61dd5d699eb4501', 'request_id': 'req-55297c54-52c9-4096-8d40-aca1ff39ecb6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.565610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] The token cache is being invalidate because project ceb80b412d08489ca61dd5d699eb4501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:00.566841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.568280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.571807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 932/1858] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:00 2022] DELETE /identity/v3/projects/ceb80b412d08489ca61dd5d699eb4501 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:00.578475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:00.581347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:00.592458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4d29840f65442885b93a4558895ddf', 'request_id': 'req-a24cae83-9d8f-4050-95eb-99883bb63f8a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.607106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4d29840f65442885b93a4558895ddf', 'request_id': 'req-a24cae83-9d8f-4050-95eb-99883bb63f8a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.615552 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4d29840f65442885b93a4558895ddf', 'request_id': 'req-a24cae83-9d8f-4050-95eb-99883bb63f8a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.635815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] The token cache is being invalidate because project 3c4d29840f65442885b93a4558895ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:00.637831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.639803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a24cae83-9d8f-4050-95eb-99883bb63f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.648374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 927/1859] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:00 2022] DELETE /identity/v3/projects/3c4d29840f65442885b93a4558895ddf => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:00.650657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8b33d-d518-45e4-bd58-d5fae821837c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.650948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8b33d-d518-45e4-bd58-d5fae821837c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.651258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8b33d-d518-45e4-bd58-d5fae821837c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.720733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6cb8b33d-d518-45e4-bd58-d5fae821837c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:00.723294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6cb8b33d-d518-45e4-bd58-d5fae821837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:00.791636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297c54-52c9-4096-8d40-aca1ff39ecb6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:00.800737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 928/1860] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:00.809898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.810277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.811381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.837018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:00.838376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:00.848869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 929/1861] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:00.853735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:00.863760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:00.864634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.865081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.865403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2590200-fb39-4938-9c60-55a763b0e8ae None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:00.865719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] PATH_INFO: `/v3/users/74b9c644b50f4b5bbfd12b069805d17a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.868576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b9c644b50f4b5bbfd12b069805d17a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:00.871417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:00.872031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:00.872791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.873658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.873658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.874115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:00.874629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:00.877513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:00.892422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 930/1862] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:38:00.896075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:00.912399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051df42-a134-4789-9501-00116bda46ac admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:00.919062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:00.920204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:00.920577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:00.920946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:00.921842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:00.924993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:00.967600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b9c644b50f4b5bbfd12b069805d17a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.982063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b9c644b50f4b5bbfd12b069805d17a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.983650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b9c644b50f4b5bbfd12b069805d17a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:00.993844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 931/1863] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:00 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:00.994775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:00.997755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:01.005575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:01.006623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b9c644b50f4b5bbfd12b069805d17a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.026155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b9c644b50f4b5bbfd12b069805d17a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.042231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 933/1864] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:00 2022] DELETE /identity/v3/users/74b9c644b50f4b5bbfd12b069805d17a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:01.171379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18f997-7375-46a4-bcc8-cd1bbd060a26 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:01.245591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:01.246567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.247059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.247601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.248726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:01.251738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:01.290901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 932/1865] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:00 2022] POST /identity/v3/users => generated 283 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:01.294693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.296780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:01.310954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca3906-8020-4213-a1dc-b8f996f1ece5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:01.315464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:01.316705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.316705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.316705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.317377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:01.319003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:01.330421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 934/1866] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:01.335160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.352724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8cdba-9e63-4e09-a61b-f25426006ad9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:01.359162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da8ba0-a4b8-4714-8f74-8e2a73e568ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:01.359799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da8ba0-a4b8-4714-8f74-8e2a73e568ba admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.360071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da8ba0-a4b8-4714-8f74-8e2a73e568ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.360699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da8ba0-a4b8-4714-8f74-8e2a73e568ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.361103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da8ba0-a4b8-4714-8f74-8e2a73e568ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:01.362711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da8ba0-a4b8-4714-8f74-8e2a73e568ba admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:01.381164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 933/1867] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:01.382855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.395354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81b894-e73b-44ec-903c-3be5aa8d6c59 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:01.400643 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:01.401646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.403356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.404060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] PATH_INFO: `/v3/projects/2426faf8e6fa489db958e447139661ad/users/985f111dce8a4970a78b707376f5f6a7/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.429605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2426faf8e6fa489db958e447139661ad, user_id=985f111dce8a4970a78b707376f5f6a7, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:01.434765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:01.472283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 935/1868] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:01 2022] PUT /identity/v3/projects/2426faf8e6fa489db958e447139661ad/users/985f111dce8a4970a78b707376f5f6a7/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:01.482656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.483177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.483177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.515600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] MFA Rules not processed for user `985f111dce8a4970a78b707376f5f6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:01.517119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:01.522743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.524725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:01.614535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2feb4b3-52ad-4d80-9806-bfabab43fc46 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:01.620640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:01.621599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.621940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.622308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] PATH_INFO: `/v3/projects/2cf4842b048840729267ceeaf383e9bb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.623771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cf4842b048840729267ceeaf383e9bb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:01.625908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:01.696634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 934/1869] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:01.697679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.710765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334e105-1c7c-4e2f-ba51-61f53ec82525 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:01.721697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cf4842b048840729267ceeaf383e9bb', 'request_id': 'req-1415b33b-0dd9-4783-88c9-cd869477a543'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.729550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cf4842b048840729267ceeaf383e9bb', 'request_id': 'req-1415b33b-0dd9-4783-88c9-cd869477a543'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.736248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:01.738010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cf4842b048840729267ceeaf383e9bb', 'request_id': 'req-1415b33b-0dd9-4783-88c9-cd869477a543'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.790908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] The token cache is being invalidate because project 2cf4842b048840729267ceeaf383e9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:01.790908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.792704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1415b33b-0dd9-4783-88c9-cd869477a543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:01.803708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 936/1870] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:01 2022] DELETE /identity/v3/projects/2cf4842b048840729267ceeaf383e9bb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:01.806892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.807355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.807729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.837084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:01.839835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:01.876525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:01.877401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:01.877823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:01.878293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:01.879641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:01.886515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:01.911636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:01.943267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 935/1871] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:01 2022] POST /identity/v3/projects => generated 367 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:01.944490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.949831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:01.958369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 937/1872] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:01.960153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:01.963874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:01.993099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c57c-d6a0-4e0d-bd09-578872e25961 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.048560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaaf81-31c6-4ce1-9a3e-3d224a160862 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.054443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.055461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.055945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.056396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] PATH_INFO: `/v3/projects/5335d3b4882446bf96e29f19dd358253` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.058077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5335d3b4882446bf96e29f19dd358253)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.060568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.074621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d197e-297a-452b-aea2-6d2d1ed44d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.075035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d197e-297a-452b-aea2-6d2d1ed44d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.075314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d197e-297a-452b-aea2-6d2d1ed44d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.075631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d197e-297a-452b-aea2-6d2d1ed44d15 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.076306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d197e-297a-452b-aea2-6d2d1ed44d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.078274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d197e-297a-452b-aea2-6d2d1ed44d15 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.108512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 938/1873] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:01 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:02.109903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf154c8-5e87-4d08-baac-9b721ad405e9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.110350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf154c8-5e87-4d08-baac-9b721ad405e9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.110459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf154c8-5e87-4d08-baac-9b721ad405e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.133086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5335d3b4882446bf96e29f19dd358253', 'request_id': 'req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:02.138973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bcf154c8-5e87-4d08-baac-9b721ad405e9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:02.140192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bcf154c8-5e87-4d08-baac-9b721ad405e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:02.140599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5335d3b4882446bf96e29f19dd358253', 'request_id': 'req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:02.146851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5335d3b4882446bf96e29f19dd358253', 'request_id': 'req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:02.165767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] The token cache is being invalidate because project 5335d3b4882446bf96e29f19dd358253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:02.166053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:02.167479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:02.172075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 936/1874] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:01 2022] DELETE /identity/v3/projects/5335d3b4882446bf96e29f19dd358253 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:02.172963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.174292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:02.209730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 939/1875] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:02.210992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.211347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.211688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.215546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:02.244869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:02.246092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:02.255330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 940/1876] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:02.256270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.261324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec18da-fbc8-404f-a0d1-3f8b0d0c787f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.266762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d71d-21b6-4b93-ba59-bbfc9f322589 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.268280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105098fc-6348-4f2b-882f-2417f2adb803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.269023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105098fc-6348-4f2b-882f-2417f2adb803 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.269278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105098fc-6348-4f2b-882f-2417f2adb803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.269551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105098fc-6348-4f2b-882f-2417f2adb803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.270257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105098fc-6348-4f2b-882f-2417f2adb803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.271437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.271905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105098fc-6348-4f2b-882f-2417f2adb803 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.272095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.272339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.272604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.273638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:02.273638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.275424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.282830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 937/1877] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:02.284172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.284461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.284743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.288456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 941/1878] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:02.289516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.301407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43839119-b3d9-4b18-97f0-e1eb51299a5a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.305744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:02.307055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:02.307460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.308286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.308722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.309168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] PATH_INFO: `/v3/projects/24195150d16642519af9f16b7c56a07f/users/adda6f9736ba4f2490ad9fd0874ee1fd/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.315582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 938/1879] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:02.316446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.328072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24195150d16642519af9f16b7c56a07f, user_id=adda6f9736ba4f2490ad9fd0874ee1fd, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.329384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741ce70-c48b-429a-a31a-62f30e65fda2 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.331156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.335147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.335768 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.336181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.337241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.338135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.340482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.351794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 942/1880] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:02 2022] PUT /identity/v3/projects/24195150d16642519af9f16b7c56a07f/users/adda6f9736ba4f2490ad9fd0874ee1fd/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:02.354025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.380927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 939/1881] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:02.382182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.387489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:02.442238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc786c4-fc8b-4ba6-a6ef-9396ffb8d706 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.506629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb16501-46cc-461d-82e5-d2b1dfa3f02d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.523659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.524541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.527270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.528450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.532046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.535491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.566974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.568798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.568798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.568798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.570151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:02.570470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.574371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.575587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 940/1882] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:02.576574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.596594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b99872-f553-4c13-bb5d-6860c5e3ba51 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.601796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.605694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.605694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.605694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.605694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.606045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 943/1883] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:02.606045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.606959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.632878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3f4e-8f94-4ced-aabc-0fe3e4e25727 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.641081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.641576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 941/1884] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/users => generated 288 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:02.642085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.642366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.642641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] PATH_INFO: `/v3/projects/24195150d16642519af9f16b7c56a07f/users/adda6f9736ba4f2490ad9fd0874ee1fd/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.643284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.645790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24195150d16642519af9f16b7c56a07f, user_id=adda6f9736ba4f2490ad9fd0874ee1fd, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.648479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.665893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162be768-8361-4788-b179-fe206cf35172 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.676089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af6be0-1102-4c95-88bb-21bbb086110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.677001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af6be0-1102-4c95-88bb-21bbb086110f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.677892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af6be0-1102-4c95-88bb-21bbb086110f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.678264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af6be0-1102-4c95-88bb-21bbb086110f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.679283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af6be0-1102-4c95-88bb-21bbb086110f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.679441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 944/1885] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:02 2022] PUT /identity/v3/projects/24195150d16642519af9f16b7c56a07f/users/adda6f9736ba4f2490ad9fd0874ee1fd/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:02.680422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.682532 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af6be0-1102-4c95-88bb-21bbb086110f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.726207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 942/1886] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:02.728198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.728721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.729096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.770070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] MFA Rules not processed for user `adda6f9736ba4f2490ad9fd0874ee1fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:02.771427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:02.788342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5003fc-8ff6-4677-8f56-38fc7e10f933 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.830079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:02.849268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.850303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.850303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.850498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.851123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.852741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.854403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:02.866296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 943/1887] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:02.867070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 945/1888] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:02.867354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.876897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.882623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4c9d0-d362-4916-857e-43605512e2be None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.888580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.889732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.889828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7dba5-84bd-41d5-a113-9ca9646eb7a2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.889915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.890085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.890916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.893015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.896013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.896814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.897332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.897666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.898455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.900202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.915326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 946/1889] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:02.916634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.918839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:02.924836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 944/1890] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:02 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:02.924836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:02.934239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697391-69ae-4d71-bd8c-52d824284a3f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:02.940434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:02.941170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:02.941449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:02.941721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] PATH_INFO: `/v3/projects/8342abc20bb04467840b9e4f8565e8d9/users/5a15faa7e83d4feaa69f62a78783e4cc/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:02.962287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8342abc20bb04467840b9e4f8565e8d9, user_id=5a15faa7e83d4feaa69f62a78783e4cc, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:02.965988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:02.993389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 945/1891] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:02 2022] PUT /identity/v3/projects/8342abc20bb04467840b9e4f8565e8d9/users/5a15faa7e83d4feaa69f62a78783e4cc/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:02.994193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:03.045683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e3cc-86be-46a7-8315-ab372486fee0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.078646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7b92b-ddf1-46c6-ae7a-24ab6266b186 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.083791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857d138-fd67-4f45-bb43-40349db05e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.084564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857d138-fd67-4f45-bb43-40349db05e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.084877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857d138-fd67-4f45-bb43-40349db05e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.085146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857d138-fd67-4f45-bb43-40349db05e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.085850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857d138-fd67-4f45-bb43-40349db05e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.088179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857d138-fd67-4f45-bb43-40349db05e13 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.100327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 946/1892] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:03.101744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.102086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.102417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.118004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.118638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.119052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.119324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.143043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] MFA Rules not processed for user `5a15faa7e83d4feaa69f62a78783e4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:03.144296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:03.221761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.223300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.244174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 947/1893] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:03.245388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:03.263216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08108562-4e76-48f0-ab1b-0859776da508 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.269481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.270110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.270323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.270648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.271648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.273797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.284900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 947/1894] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:03.298173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:03.301353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:03.304692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 948/1895] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:03.305902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:03.345653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:03.346929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653523b-9bc9-4882-85eb-07bcc2f6a38e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.362802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.376902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.377422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.378672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] PATH_INFO: `/v3/projects/774c6f6d97d0487284f5c6eca43cb6bd/users/3e033e2e5e194796a4b743dcf236de02/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.428022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774c6f6d97d0487284f5c6eca43cb6bd, user_id=3e033e2e5e194796a4b743dcf236de02, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.434099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.450023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b63a6f-43eb-4bae-95df-7544f6988f34 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.470168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 949/1896] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:03 2022] PUT /identity/v3/projects/774c6f6d97d0487284f5c6eca43cb6bd/users/3e033e2e5e194796a4b743dcf236de02/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:03.472002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:03.496102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:03.530216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3766e-01d1-491a-b71e-877f9f8fb779 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.530838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3766e-01d1-491a-b71e-877f9f8fb779 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.531112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3766e-01d1-491a-b71e-877f9f8fb779 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.531385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3766e-01d1-491a-b71e-877f9f8fb779 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.574559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd3740-bba2-4367-8dec-ec86c2ded336 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.580151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.580825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.581341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.581621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.585135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:03.610939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3766e-01d1-491a-b71e-877f9f8fb779 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.612534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3766e-01d1-491a-b71e-877f9f8fb779 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.622706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 948/1897] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:03.623991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.624338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.624683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.658356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] MFA Rules not processed for user `3e033e2e5e194796a4b743dcf236de02`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:03.659635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:03.661118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.662889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.673504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 950/1898] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:03.749646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 949/1899] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:03.754199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:03.891558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddd440-c672-4dcf-9c3e-94edb3d0b747 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:03.904238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:03.905635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:03.905635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:03.917468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:03.918344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:03.922793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:03.969639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 951/1900] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:03 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:03.975171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.002411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683e112-b30a-45f1-87fb-f0eadb4796aa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.010473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:04.109525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.110294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.110639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.111206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.125566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.134735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:04.181056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 950/1901] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:03 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:04.184816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.201578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfa4f3-a88a-4b2c-bf52-0995f3d11cab admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.213672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.215178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.215494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.231629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.232680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.235534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:04.252342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 952/1902] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:04.257261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.272433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4048564-99b8-4795-a91f-3e1929ff08fd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.290610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.291316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.291593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.299483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] PATH_INFO: `/v3/projects/61b33c03e2924d41aee5bdcbf2817a99/users/2852d0330a084bfaa51e92da14208519/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.329602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b33c03e2924d41aee5bdcbf2817a99, user_id=2852d0330a084bfaa51e92da14208519, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.333907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:04.358481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 951/1903] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:04 2022] PUT /identity/v3/projects/61b33c03e2924d41aee5bdcbf2817a99/users/2852d0330a084bfaa51e92da14208519/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:04.362564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.378145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d768e-b6e9-4376-ac06-f100bcaa609d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.488460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:04.527165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28fe69-9b12-4b32-a8d2-3654bf092ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.537286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28fe69-9b12-4b32-a8d2-3654bf092ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.538227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28fe69-9b12-4b32-a8d2-3654bf092ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.538627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28fe69-9b12-4b32-a8d2-3654bf092ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.540717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28fe69-9b12-4b32-a8d2-3654bf092ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.545066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28fe69-9b12-4b32-a8d2-3654bf092ec2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:04.569632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 953/1904] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:04.569846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.583136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7ffd3-7406-48da-8a0e-1533682e4523 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.600277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.601013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.601291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.601570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] PATH_INFO: `/v3/projects/61b33c03e2924d41aee5bdcbf2817a99/users/2852d0330a084bfaa51e92da14208519/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.603745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b33c03e2924d41aee5bdcbf2817a99, user_id=2852d0330a084bfaa51e92da14208519, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.606461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:04.623468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 952/1905] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:04 2022] PUT /identity/v3/projects/61b33c03e2924d41aee5bdcbf2817a99/users/2852d0330a084bfaa51e92da14208519/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:04.634556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.634838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.635110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.671671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] MFA Rules not processed for user `2852d0330a084bfaa51e92da14208519`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:04.673018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:04.759536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 954/1906] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:04.768231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.770569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:04.835600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31092a-dc9e-41a9-b49d-80cf1b3aca1a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.839776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeceb03d-c5f4-4a5d-ac8f-b4752e85dd7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.840429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceb03d-c5f4-4a5d-ac8f-b4752e85dd7a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.840702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceb03d-c5f4-4a5d-ac8f-b4752e85dd7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.841052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceb03d-c5f4-4a5d-ac8f-b4752e85dd7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.852117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeceb03d-c5f4-4a5d-ac8f-b4752e85dd7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.853585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeceb03d-c5f4-4a5d-ac8f-b4752e85dd7a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:04.863546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 953/1907] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:04.900628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:04.971402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e24a8-efc5-46fd-9ed3-8d3b2ef176d4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:04.977504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:04.978103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:04.978435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:04.978765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] PATH_INFO: `/v3/users/3e033e2e5e194796a4b743dcf236de02` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:04.980193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e033e2e5e194796a4b743dcf236de02)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:04.982475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:05.055441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e033e2e5e194796a4b743dcf236de02'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.067503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e033e2e5e194796a4b743dcf236de02'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.068238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e033e2e5e194796a4b743dcf236de02'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.094312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:05.099097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e033e2e5e194796a4b743dcf236de02'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.116185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e033e2e5e194796a4b743dcf236de02'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.123229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 955/1908] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:04 2022] DELETE /identity/v3/users/3e033e2e5e194796a4b743dcf236de02 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:05.128526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf24dae-e175-4607-b124-493c086e262d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:05.129298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf24dae-e175-4607-b124-493c086e262d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:05.129578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf24dae-e175-4607-b124-493c086e262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:05.155654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-caf24dae-e175-4607-b124-493c086e262d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:05.157172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-caf24dae-e175-4607-b124-493c086e262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:05.254770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 954/1909] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:05.264790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:05.266342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:05.366352 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4ebe0-242b-4907-a5e0-d76110210bb5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:05.377495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:05.378265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:05.379348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:05.379723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:05.391494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:05.395326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:05.409995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 956/1910] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:05.763081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf24dae-e175-4607-b124-493c086e262d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:05.766084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-caf24dae-e175-4607-b124-493c086e262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:05.775240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:05.782425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf24dae-e175-4607-b124-493c086e262d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:05.789807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:05.790788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:05.791125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:05.791460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] PATH_INFO: `/v3/projects/774c6f6d97d0487284f5c6eca43cb6bd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:05.792837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774c6f6d97d0487284f5c6eca43cb6bd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:05.795575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:05.859759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774c6f6d97d0487284f5c6eca43cb6bd', 'request_id': 'req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.867225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774c6f6d97d0487284f5c6eca43cb6bd', 'request_id': 'req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.894388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774c6f6d97d0487284f5c6eca43cb6bd', 'request_id': 'req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.917471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] The token cache is being invalidate because project 774c6f6d97d0487284f5c6eca43cb6bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:05.918229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.919610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:05.928697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 955/1911] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:05 2022] DELETE /identity/v3/projects/774c6f6d97d0487284f5c6eca43cb6bd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:05.935814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:05.952332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28746e-a79a-4074-971c-fb9ce7bc2392 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:06.071558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:06.072266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.072551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.072869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] PATH_INFO: `/v3/users/2852d0330a084bfaa51e92da14208519` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.074062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2852d0330a084bfaa51e92da14208519)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:06.076151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:06.127473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2852d0330a084bfaa51e92da14208519'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.136074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2852d0330a084bfaa51e92da14208519'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.136708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2852d0330a084bfaa51e92da14208519'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.155633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2852d0330a084bfaa51e92da14208519'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.162801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0b1ec138-98e2-4a98-86c3-b9fbd1b54499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2852d0330a084bfaa51e92da14208519'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.169552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 957/1912] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:05 2022] DELETE /identity/v3/users/2852d0330a084bfaa51e92da14208519 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:06.480303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:06.481938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:06.497469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bb8cd-8e40-488d-b5b2-f2212edabb10 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:06.556209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:06.556891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.557320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.557475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] PATH_INFO: `/v3/projects/61b33c03e2924d41aee5bdcbf2817a99` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.558670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b33c03e2924d41aee5bdcbf2817a99)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:06.563167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:06.613460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b33c03e2924d41aee5bdcbf2817a99', 'request_id': 'req-b5361579-4f4e-48ed-be71-4186f2815251'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.620485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b33c03e2924d41aee5bdcbf2817a99', 'request_id': 'req-b5361579-4f4e-48ed-be71-4186f2815251'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.627186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b33c03e2924d41aee5bdcbf2817a99', 'request_id': 'req-b5361579-4f4e-48ed-be71-4186f2815251'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.643074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] The token cache is being invalidate because project 61b33c03e2924d41aee5bdcbf2817a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:06.643386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.644051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b5361579-4f4e-48ed-be71-4186f2815251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:06.645867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 956/1913] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:06 2022] DELETE /identity/v3/projects/61b33c03e2924d41aee5bdcbf2817a99 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:06.650867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.651408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.651494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.665454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:06.675117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:06.677028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:06.739584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 958/1914] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:06.748695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.749014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.749295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.770143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:06.771365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:06.780518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 957/1915] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:06.785521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:06.796334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0c848-2e98-41a8-a428-e2fadcb5b6da None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:06.801584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:06.802166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.802464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.802735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.803441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:06.803741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:06.805403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:06.817590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 959/1916] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:06.822373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:06.832180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37beb56-f323-430d-9e20-ff9099c140d3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:06.838522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:06.839144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.839421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.839682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.840354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:06.842287 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:06.848469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:06.873865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 958/1917] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:06 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:06.878186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:06.890538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2fab-9781-43a7-a682-17cdfd7c9bc5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:06.948205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:06.948893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:06.949170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:06.949446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:06.950095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:06.952172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:06.974699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 960/1918] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:06 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:06.978809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:06.995748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62927101-bea9-4257-9f94-afd952618cda admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.002043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.002667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.002938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.003201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.004249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.005999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.023086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 959/1919] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:07.029898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.043641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22858b4-3151-4783-9c55-671ce69b3db6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.048353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97143aa7-0dbd-4c7a-8e9c-4b2c554aa098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.049641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97143aa7-0dbd-4c7a-8e9c-4b2c554aa098 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.049920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97143aa7-0dbd-4c7a-8e9c-4b2c554aa098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.050188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97143aa7-0dbd-4c7a-8e9c-4b2c554aa098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.050879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97143aa7-0dbd-4c7a-8e9c-4b2c554aa098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.052469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97143aa7-0dbd-4c7a-8e9c-4b2c554aa098 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.064273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 961/1920] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:07.068478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.079643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6ee0-3f1e-466e-ac48-ecd94eedf3dc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.085242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.085966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.086245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.086548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] PATH_INFO: `/v3/projects/04ed57ee5fb2403fa92e149a8f3ddbda/users/9837705728304da48d3994095c7e6ba3/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.107317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ed57ee5fb2403fa92e149a8f3ddbda, user_id=9837705728304da48d3994095c7e6ba3, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.109423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.146009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 960/1921] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:07 2022] PUT /identity/v3/projects/04ed57ee5fb2403fa92e149a8f3ddbda/users/9837705728304da48d3994095c7e6ba3/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:07.146195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.146195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.146195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.189843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] MFA Rules not processed for user `9837705728304da48d3994095c7e6ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:07.198274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:07.321000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 962/1922] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:07.324305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.426133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e005dfa-c174-49a6-842b-6c6a87d28116 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.431453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.431590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.431836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.432110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.432886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.437933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.472984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 961/1923] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:07 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:07.477379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.483316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:07.501834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fad1b-7565-45da-b79c-ac2379f6df81 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.619904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.620894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.621178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.621440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.622282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.624649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.679035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 963/1924] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:07 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:07.680754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.691204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b8c91-4926-4f09-993a-5a0d43959d73 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.696905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.697573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.697860 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.698117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.698954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.700698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.712186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 962/1925] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:07.716657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.726683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2d15d-02ec-4257-b4bc-e739447fc454 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.731229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.732248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.732363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.732597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] PATH_INFO: `/v3/projects/5977726e9f2c4f13ac6237fadd09f736/users/51fd0d41bde748e6a239f2e1567653f7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.751623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5977726e9f2c4f13ac6237fadd09f736, user_id=51fd0d41bde748e6a239f2e1567653f7, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.754817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.770745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 964/1926] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:07 2022] PUT /identity/v3/projects/5977726e9f2c4f13ac6237fadd09f736/users/51fd0d41bde748e6a239f2e1567653f7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:07.774618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.785309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245cc02-3587-4a10-a0a9-95c12d9ec9bd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.849428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2874563-807e-4c57-b18f-f34b2b443f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.850204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874563-807e-4c57-b18f-f34b2b443f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.850590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874563-807e-4c57-b18f-f34b2b443f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.850953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874563-807e-4c57-b18f-f34b2b443f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.851882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2874563-807e-4c57-b18f-f34b2b443f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.853582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2874563-807e-4c57-b18f-f34b2b443f63 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.866659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 963/1927] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:07.867791 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:07.877991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e9594-ba1e-4254-9b76-58cc5dde5d61 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:07.883604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:07.886967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.886967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.886967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] PATH_INFO: `/v3/projects/5977726e9f2c4f13ac6237fadd09f736/users/51fd0d41bde748e6a239f2e1567653f7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.886967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5977726e9f2c4f13ac6237fadd09f736, user_id=51fd0d41bde748e6a239f2e1567653f7, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:07.889230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:07.901046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 965/1928] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:07 2022] PUT /identity/v3/projects/5977726e9f2c4f13ac6237fadd09f736/users/51fd0d41bde748e6a239f2e1567653f7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:07.912079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:07.912370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:07.912688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:07.945349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] MFA Rules not processed for user `51fd0d41bde748e6a239f2e1567653f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:07.946809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:07.959025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:08.038417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 964/1929] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:08.052487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:08.055351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:08.117399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4f0e3-024f-4fae-b10a-fd0ffbb66035 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:08.121408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:08.122025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:08.122363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:08.122632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:08.132170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:08.133681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:08.144086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 966/1930] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:08.264078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:08.275496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84091-f853-4f13-a0b5-3863211e17e3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:08.279875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d3526-5c71-4d42-8688-b3f3747af09a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:08.280518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3526-5c71-4d42-8688-b3f3747af09a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:08.280847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3526-5c71-4d42-8688-b3f3747af09a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:08.281119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d3526-5c71-4d42-8688-b3f3747af09a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:08.345239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d3526-5c71-4d42-8688-b3f3747af09a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:08.346818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d3526-5c71-4d42-8688-b3f3747af09a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:08.357336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 965/1931] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:09.200402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:09.202354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:09.266865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:09.272031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a172-e657-4e9c-837b-52ec73fbdbca service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:09.277667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:09.278888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:09.279182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:09.279501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] PATH_INFO: `/v3/users/985f111dce8a4970a78b707376f5f6a7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:09.280740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985f111dce8a4970a78b707376f5f6a7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:09.282781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:09.340204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985f111dce8a4970a78b707376f5f6a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:09.348016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985f111dce8a4970a78b707376f5f6a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:09.348799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985f111dce8a4970a78b707376f5f6a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:09.363319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985f111dce8a4970a78b707376f5f6a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:09.371668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985f111dce8a4970a78b707376f5f6a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:09.379063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 967/1932] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:09 2022] DELETE /identity/v3/users/985f111dce8a4970a78b707376f5f6a7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:09.383944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:09.384521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:09.384521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:09.403346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:09.404678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:09.465247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 966/1933] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:09.475358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:09.478528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:09.571346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f27c87-6b97-4971-a468-aa255cee5491 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:09.575745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:09.576574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:09.576878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:09.577147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:09.587092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:09.588602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:09.601691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 968/1934] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:10.026690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:10.030216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:10.056388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c818ae-91f7-40b2-bdc2-9c57aabdc242 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:10.061250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:10.061947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:10.062225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:10.062491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] PATH_INFO: `/v3/projects/2426faf8e6fa489db958e447139661ad` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:10.063836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2426faf8e6fa489db958e447139661ad)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:10.066064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:10.115121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:10.123224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2426faf8e6fa489db958e447139661ad', 'request_id': 'req-f972d5b0-46fd-4572-befd-6040f179157b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.131605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2426faf8e6fa489db958e447139661ad', 'request_id': 'req-f972d5b0-46fd-4572-befd-6040f179157b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.139527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2426faf8e6fa489db958e447139661ad', 'request_id': 'req-f972d5b0-46fd-4572-befd-6040f179157b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.171826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] The token cache is being invalidate because project 2426faf8e6fa489db958e447139661ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:10.172225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.175773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.177880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 967/1935] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:10 2022] DELETE /identity/v3/projects/2426faf8e6fa489db958e447139661ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:10.189185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:10.204081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2447fda-6409-46bb-b3c1-5d36631b878e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:10.277404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:10.278496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:10.278969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:10.279407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] PATH_INFO: `/v3/users/adda6f9736ba4f2490ad9fd0874ee1fd` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:10.280830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adda6f9736ba4f2490ad9fd0874ee1fd)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:10.283097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:10.315165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:10.353798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adda6f9736ba4f2490ad9fd0874ee1fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.366154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adda6f9736ba4f2490ad9fd0874ee1fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.367857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adda6f9736ba4f2490ad9fd0874ee1fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.384855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adda6f9736ba4f2490ad9fd0874ee1fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.395587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4171b114-7b7a-40b9-94ed-119029c205a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adda6f9736ba4f2490ad9fd0874ee1fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.403710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 969/1936] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:10 2022] DELETE /identity/v3/users/adda6f9736ba4f2490ad9fd0874ee1fd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:10.730747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:10.733027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:10.757805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d5b0-46fd-4572-befd-6040f179157b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:10.867577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:10.868462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:10.868916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:10.869321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] PATH_INFO: `/v3/projects/24195150d16642519af9f16b7c56a07f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:10.871108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24195150d16642519af9f16b7c56a07f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:10.873434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:10.952864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24195150d16642519af9f16b7c56a07f', 'request_id': 'req-6d4b07cd-c381-4546-8f44-d876531099f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.952864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24195150d16642519af9f16b7c56a07f', 'request_id': 'req-6d4b07cd-c381-4546-8f44-d876531099f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.962983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24195150d16642519af9f16b7c56a07f', 'request_id': 'req-6d4b07cd-c381-4546-8f44-d876531099f5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.983082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] The token cache is being invalidate because project 24195150d16642519af9f16b7c56a07f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:10.983503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.984567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6d4b07cd-c381-4546-8f44-d876531099f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:10.986759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 968/1937] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:10 2022] DELETE /identity/v3/projects/24195150d16642519af9f16b7c56a07f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:10.991728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:10.991997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:10.992273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.018236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:11.019743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.095263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 970/1938] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:11.103417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.103795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.104077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.124090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.125685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.138981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f808e05-a7d8-4ee2-9fb0-6a5475f75b55 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.139624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:11.140965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.144375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.145274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.145754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.146167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] PATH_INFO: `/v3/users/9837705728304da48d3994095c7e6ba3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.147510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9837705728304da48d3994095c7e6ba3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.149644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.155158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 969/1939] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:11.156230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.166911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7e41-de3b-417e-8706-915c1e4cb195 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.171817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.172447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.172904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.173169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.174615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:11.174615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.175796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.189183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 970/1940] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:38:11.193651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.206675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4fef5-1c45-4f71-9727-241d330049d6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.212958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.213848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.214063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.214337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.215021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.217995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.226140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9837705728304da48d3994095c7e6ba3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:11.242953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9837705728304da48d3994095c7e6ba3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:11.243751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9837705728304da48d3994095c7e6ba3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:11.259786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 971/1941] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:11 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:11.260740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.264146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9837705728304da48d3994095c7e6ba3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:11.272608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2af08f1b-33a6-4172-bf77-4c8922b1b502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9837705728304da48d3994095c7e6ba3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:11.281006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 971/1942] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:11 2022] DELETE /identity/v3/users/9837705728304da48d3994095c7e6ba3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:11.285213 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.285600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.286331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.288884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:11.311172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:11.312537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.318045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:11.347571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e28b8-d7e3-4eb6-9a80-f0273c481acc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.399523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 972/1943] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:11.411040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.414054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.428028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.429770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.430237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.430641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.431775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.435960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.484058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 972/1944] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:11 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:11.485733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.487903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.492679 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b76ac-6e66-4f32-b805-889758e0de2d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.497137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.497821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.498120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.498407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.501747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed5d28-bcc5-434d-8f7d-c4ed33c63d93 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.507563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.508315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.509030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.509429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.509809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.510615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.510726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.513260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.526067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 973/1945] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:11.536926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 973/1946] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:11.541349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.554172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0f61c-9f99-4139-a047-a18370d82703 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.560444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5acf51-b732-4b6e-a114-c5d58b3af59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.561734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5acf51-b732-4b6e-a114-c5d58b3af59f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.561734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5acf51-b732-4b6e-a114-c5d58b3af59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.561951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5acf51-b732-4b6e-a114-c5d58b3af59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.562516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5acf51-b732-4b6e-a114-c5d58b3af59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.564542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5acf51-b732-4b6e-a114-c5d58b3af59f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.580925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 974/1947] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:11.585434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.601927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a61a5-3638-4ba3-9bcb-bbb23e9d0a82 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.609318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.610479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.611081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.611500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] PATH_INFO: `/v3/projects/1bc3a8fad6f941619b2f320f601cfef0/users/4d578e1bcd54432289fa91e088246252/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.643336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc3a8fad6f941619b2f320f601cfef0, user_id=4d578e1bcd54432289fa91e088246252, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:11.647506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:11.666079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 974/1948] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:11 2022] PUT /identity/v3/projects/1bc3a8fad6f941619b2f320f601cfef0/users/4d578e1bcd54432289fa91e088246252/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:11.676647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.676988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.677266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:11.738004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] MFA Rules not processed for user `4d578e1bcd54432289fa91e088246252`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:11.739925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.838336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 975/1949] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:11.842694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.903564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:11.905207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:11.972930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1c12-c0aa-4019-b06c-6759cda67983 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:11.987766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:11.990691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:11.991591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:11.992985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.001684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:12.014575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.042875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.071383 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764446-f956-43af-8152-1deb00e08e76 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.080808 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.081456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.081688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.081962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] PATH_INFO: `/v3/projects/04ed57ee5fb2403fa92e149a8f3ddbda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.084610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ed57ee5fb2403fa92e149a8f3ddbda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.091799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.122802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 975/1950] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:11 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:12.129459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.146312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:12.180293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ed57ee5fb2403fa92e149a8f3ddbda', 'request_id': 'req-e8b41e70-4e0b-4a99-a4f8-0763494697b5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.187364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ed57ee5fb2403fa92e149a8f3ddbda', 'request_id': 'req-e8b41e70-4e0b-4a99-a4f8-0763494697b5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.193402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ed57ee5fb2403fa92e149a8f3ddbda', 'request_id': 'req-e8b41e70-4e0b-4a99-a4f8-0763494697b5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.213062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] The token cache is being invalidate because project 04ed57ee5fb2403fa92e149a8f3ddbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:12.213506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.215270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.221306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 976/1951] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:11 2022] DELETE /identity/v3/projects/04ed57ee5fb2403fa92e149a8f3ddbda => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:12.223582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.235747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:12.277966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca8e42-dc6d-4846-abcf-8ac3e403c259 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.329324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b41e70-4e0b-4a99-a4f8-0763494697b5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.335365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.336393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.336897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.337159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] PATH_INFO: `/v3/users/51fd0d41bde748e6a239f2e1567653f7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.338533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51fd0d41bde748e6a239f2e1567653f7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.340580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.349528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.350143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.350428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.350702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.351564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.356502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.393087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 976/1952] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:12 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:12.397264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.400334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:12.415144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e194d-7e6d-47e6-b6d0-c6cb8cf5f953 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.416143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51fd0d41bde748e6a239f2e1567653f7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.420632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.421300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.421575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.421843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.422599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.424616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.430004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51fd0d41bde748e6a239f2e1567653f7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.430837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51fd0d41bde748e6a239f2e1567653f7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.437947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 977/1953] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:12.442096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.447211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:12.453752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c386d84-9507-4c62-8fc8-6768de8d2c36 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.459614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.460504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.460863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.461265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] PATH_INFO: `/v3/projects/a24284e225c34199809b213bae06e3a4/users/1887969b061940e29ec0045eef85e639/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.463381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51fd0d41bde748e6a239f2e1567653f7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.476007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51fd0d41bde748e6a239f2e1567653f7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:12.482833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24284e225c34199809b213bae06e3a4, user_id=1887969b061940e29ec0045eef85e639, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.485279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 977/1954] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:12 2022] DELETE /identity/v3/users/51fd0d41bde748e6a239f2e1567653f7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:12.488318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.511396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 978/1955] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:12 2022] PUT /identity/v3/projects/a24284e225c34199809b213bae06e3a4/users/1887969b061940e29ec0045eef85e639/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:12.515167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.517234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:12.605136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f2864-3b6c-43c2-a87b-e1f7ad041747 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.714226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8e29-e4a1-44be-b1b7-ff2c28068eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.715309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8e29-e4a1-44be-b1b7-ff2c28068eae admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.715309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8e29-e4a1-44be-b1b7-ff2c28068eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.715513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8e29-e4a1-44be-b1b7-ff2c28068eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.716386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8e29-e4a1-44be-b1b7-ff2c28068eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.718825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8e29-e4a1-44be-b1b7-ff2c28068eae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.733864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 978/1956] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:12.738172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.755495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1db208-8433-41ea-88b8-b8dfcaeb274f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:12.766695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:12.769104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.773205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.773540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] PATH_INFO: `/v3/projects/a24284e225c34199809b213bae06e3a4/users/1887969b061940e29ec0045eef85e639/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.776724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24284e225c34199809b213bae06e3a4, user_id=1887969b061940e29ec0045eef85e639, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:12.785075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:12.809927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 979/1957] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:12 2022] PUT /identity/v3/projects/a24284e225c34199809b213bae06e3a4/users/1887969b061940e29ec0045eef85e639/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:12.822633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:12.823059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:12.823520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:12.868146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:12.871516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:12.890778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] MFA Rules not processed for user `1887969b061940e29ec0045eef85e639`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:12.895165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:12.973991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:12.993998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46c1a2-1901-4593-b024-c15af6cc9f22 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.001580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.003098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.017601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.018138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] PATH_INFO: `/v3/projects/5977726e9f2c4f13ac6237fadd09f736` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.019686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5977726e9f2c4f13ac6237fadd09f736)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.022158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.059150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 979/1958] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:13.074922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.076274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:13.099675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:13.113070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5977726e9f2c4f13ac6237fadd09f736', 'request_id': 'req-b6609a03-720a-49c9-9920-144af115a59e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:13.126506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5977726e9f2c4f13ac6237fadd09f736', 'request_id': 'req-b6609a03-720a-49c9-9920-144af115a59e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:13.137309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5977726e9f2c4f13ac6237fadd09f736', 'request_id': 'req-b6609a03-720a-49c9-9920-144af115a59e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:13.162583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] The token cache is being invalidate because project 5977726e9f2c4f13ac6237fadd09f736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:13.162869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:13.163552 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b6609a03-720a-49c9-9920-144af115a59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:13.165125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 980/1959] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:12 2022] DELETE /identity/v3/projects/5977726e9f2c4f13ac6237fadd09f736 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:13.168819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dc089-b001-4609-b906-d0f1d5571760 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.170312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3339e-bc44-4968-9a15-33b9c976a53b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.170622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3339e-bc44-4968-9a15-33b9c976a53b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.171043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3339e-bc44-4968-9a15-33b9c976a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.192564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ecd3339e-bc44-4968-9a15-33b9c976a53b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:13.193831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ecd3339e-bc44-4968-9a15-33b9c976a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:13.220912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.221594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.222484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.222484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.233979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:13.262408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 981/1960] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:13.274361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.274658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.274979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.295718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:13.298851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:13.311250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 982/1961] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:13.313467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.316654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.319064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.323493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c97722-0cd9-405e-9b71-7396e4125b88 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.332275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.333130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.333408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.333677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.335330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:13.335733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.337630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.339182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 980/1962] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:13.364006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 983/1963] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:13.365766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.384223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18679-db71-4523-bfd1-2317ead94421 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.393388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.394041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.394313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.394582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.395256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.397453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.454445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 981/1964] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:13 2022] POST /identity/v3/projects => generated 389 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:13.464988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.477225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.478981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:13.486875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6ae21-b702-4db7-830d-2380ce35ddcc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.578547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.579557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.579982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.580426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.581403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.585072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.592292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bb999-f35f-40cf-a62d-d8e602871573 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.604590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.605283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.605560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.605830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.607409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:13.608158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:13.621382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:13.632867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 984/1965] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:13 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:13.637740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.649992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e1d4b-8923-49f1-bd78-55e9e368e9c9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.657454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.658761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.659033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.659594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.660356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.663082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.684166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 985/1966] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:13.685595 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.696672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.697577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426df2ba-96be-40bd-af2d-662cca120c1b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.698233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.702031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b214513-f896-4643-a6d9-61bb4d430360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.702875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b214513-f896-4643-a6d9-61bb4d430360 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.703218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b214513-f896-4643-a6d9-61bb4d430360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.704011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b214513-f896-4643-a6d9-61bb4d430360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.704975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b214513-f896-4643-a6d9-61bb4d430360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.707127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b214513-f896-4643-a6d9-61bb4d430360 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.710678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 982/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:13.723362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 986/1968] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:13.727374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:13.741561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91665924-0498-4229-981f-dbaccc85dc00 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:13.755080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:13.755721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.756032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.756263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] PATH_INFO: `/v3/projects/c063e1aba4aa4187b3e6166a9ac793f0/users/84944797e82342579246ea30add6cb51/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.788831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c063e1aba4aa4187b3e6166a9ac793f0, user_id=84944797e82342579246ea30add6cb51, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:13.793979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:13.817313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 983/1969] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:13 2022] PUT /identity/v3/projects/c063e1aba4aa4187b3e6166a9ac793f0/users/84944797e82342579246ea30add6cb51/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:13.828218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:13.829247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:13.829918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:13.901305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] MFA Rules not processed for user `84944797e82342579246ea30add6cb51`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:13.903088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:14.035904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 987/1970] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:14.040962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.150145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.151546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3541-beb8-46de-be16-ce95341fb0a6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.151733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:14.187812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.188561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.188942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.192906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.193807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.194241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6935d68-284d-4802-b665-6ef8710375fe None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.196933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.225182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.225855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.226234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.227108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] PATH_INFO: `/v3/users/4d578e1bcd54432289fa91e088246252` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.228623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d578e1bcd54432289fa91e088246252)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.231881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.266948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 984/1971] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:14 2022] POST /identity/v3/projects => generated 389 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:14.283201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.318215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d578e1bcd54432289fa91e088246252'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:14.329206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d578e1bcd54432289fa91e088246252'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:14.329997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d578e1bcd54432289fa91e088246252'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:14.339224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d578e1bcd54432289fa91e088246252'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:14.340093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8b76b9b-e8d7-4de0-8dae-888296f8600e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d578e1bcd54432289fa91e088246252'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:14.377590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 988/1972] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:14 2022] DELETE /identity/v3/users/4d578e1bcd54432289fa91e088246252 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:14.380168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.380535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.380915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.391819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf17512-f4cf-478b-b372-2923b90edb17 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.417727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:14.424816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:14.427012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:14.491350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:14.511297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.511958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.512239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.512772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.513654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.516195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.544016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 985/1973] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:14 2022] POST /identity/v3/users => generated 301 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:14.547777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.549496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:14.564159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4681856-85bf-4d51-a2d0-15ed372f4a0a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.566017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 989/1974] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:14.570318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.570970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.571250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.571524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.572225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.573867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.584749 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 986/1975] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:14.589639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.597064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.599006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:14.601119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce622b-ebc9-4da9-94d0-59615469faee None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.607229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.608039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.608423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.608835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] PATH_INFO: `/v3/projects/46e947d99f68495aa555eafe45cbb6f2/users/c19d0f1cd5e3420aa1ddf3fe3deaffd4/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.627942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e947d99f68495aa555eafe45cbb6f2, user_id=c19d0f1cd5e3420aa1ddf3fe3deaffd4, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.631545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.658857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 990/1976] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:14 2022] PUT /identity/v3/projects/46e947d99f68495aa555eafe45cbb6f2/users/c19d0f1cd5e3420aa1ddf3fe3deaffd4/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:14.660412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.681127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a4f2b-242b-4cfe-9809-bde09371ec60 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.739347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fafbb-5c05-4f23-b6b1-f6dabeac0c2d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.739725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c4e68-634c-40d8-9c03-6bab28c09cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.741219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c4e68-634c-40d8-9c03-6bab28c09cea service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.741577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c4e68-634c-40d8-9c03-6bab28c09cea service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.741823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c4e68-634c-40d8-9c03-6bab28c09cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.744418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.745273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.745376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.745617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.746319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.748136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.751985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c4e68-634c-40d8-9c03-6bab28c09cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.760253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 991/1977] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:14.761600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c4e68-634c-40d8-9c03-6bab28c09cea service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.764546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.774652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 987/1978] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:14.782827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae885d9-a402-4493-a5ab-8350a8c11078 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:14.787998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:14.788685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.789143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.789437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] PATH_INFO: `/v3/projects/46e947d99f68495aa555eafe45cbb6f2/users/c19d0f1cd5e3420aa1ddf3fe3deaffd4/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.792144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e947d99f68495aa555eafe45cbb6f2, user_id=c19d0f1cd5e3420aa1ddf3fe3deaffd4, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:14.795070 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:14.806908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 992/1979] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:14 2022] PUT /identity/v3/projects/46e947d99f68495aa555eafe45cbb6f2/users/c19d0f1cd5e3420aa1ddf3fe3deaffd4/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:14.819009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:14.819355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:14.819545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:14.848322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] MFA Rules not processed for user `c19d0f1cd5e3420aa1ddf3fe3deaffd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:14.849726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:14.906143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:14.946925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 988/1980] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:14.957994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:14.961729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:15.038117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa201795-0afd-4ab4-a677-f50243a2665c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:15.042272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:15.042787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:15.042911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:15.043102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:15.056501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:15.058249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:15.073373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 993/1981] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:15.075112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:15.076536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:15.186716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9e419-a855-4d1b-8a36-02fdbcf0631b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:15.191782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:15.192525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:15.192919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:15.193259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] PATH_INFO: `/v3/projects/1bc3a8fad6f941619b2f320f601cfef0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:15.194728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc3a8fad6f941619b2f320f601cfef0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:15.196628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:15.241816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:15.253110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc3a8fad6f941619b2f320f601cfef0', 'request_id': 'req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.253342 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a514717-d3b2-4ef7-be82-dc89dc2409ff service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:15.259889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc3a8fad6f941619b2f320f601cfef0', 'request_id': 'req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.265364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc3a8fad6f941619b2f320f601cfef0', 'request_id': 'req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.279228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] The token cache is being invalidate because project 1bc3a8fad6f941619b2f320f601cfef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:15.279697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.280320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.281987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 989/1982] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:15 2022] DELETE /identity/v3/projects/1bc3a8fad6f941619b2f320f601cfef0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:15.290572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:15.292116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:15.293095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:15.327982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:15.328989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:15.329447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:15.329978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] PATH_INFO: `/v3/users/84944797e82342579246ea30add6cb51` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:15.331302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84944797e82342579246ea30add6cb51)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:15.333598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:15.360722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafd5ae-34bf-473d-ba0d-69b1516e3b5e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:15.365758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:15.366477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:15.366782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:15.367045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] PATH_INFO: `/v3/users/1887969b061940e29ec0045eef85e639` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:15.368230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1887969b061940e29ec0045eef85e639)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:15.370255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:15.384736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:15.386316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84944797e82342579246ea30add6cb51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.395005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84944797e82342579246ea30add6cb51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.395761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84944797e82342579246ea30add6cb51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.405248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84944797e82342579246ea30add6cb51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.406112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3abf823a-682e-4769-b9ad-50bfc6c39611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84944797e82342579246ea30add6cb51'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.420719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1887969b061940e29ec0045eef85e639'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.425509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 994/1983] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:15 2022] DELETE /identity/v3/users/84944797e82342579246ea30add6cb51 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:15.428384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1887969b061940e29ec0045eef85e639'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.429369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1887969b061940e29ec0045eef85e639'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.429960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:15.430263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:15.430525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:15.442363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1887969b061940e29ec0045eef85e639'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.449122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1887969b061940e29ec0045eef85e639'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:15.449936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:15.451197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:15.455380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 990/1984] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:15 2022] DELETE /identity/v3/users/1887969b061940e29ec0045eef85e639 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:15.517895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 995/1985] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:15.528059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:15.529626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:15.599520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45eabab-421e-42c8-9c64-e1a16dcb6591 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:15.603499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:15.604227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:15.604506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:15.604788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:15.673883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:15.675424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:15.686916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 991/1986] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:16.094579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.096137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.109359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1dcc-1694-4893-86d4-406651542be6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.112498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.114518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.115673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.116542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.117086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.117458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] PATH_INFO: `/v3/projects/a24284e225c34199809b213bae06e3a4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.118753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24284e225c34199809b213bae06e3a4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.121611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.130985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cab2dc-af38-4eac-9376-2da2ed403302 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.137865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.138578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.138853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.139126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] PATH_INFO: `/v3/projects/c063e1aba4aa4187b3e6166a9ac793f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.140408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c063e1aba4aa4187b3e6166a9ac793f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.142947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.191351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:16.194384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24284e225c34199809b213bae06e3a4', 'request_id': 'req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.195700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:16.201841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24284e225c34199809b213bae06e3a4', 'request_id': 'req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.203514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c063e1aba4aa4187b3e6166a9ac793f0', 'request_id': 'req-5216022b-7b7f-4276-8d62-3b988ca4f81a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.213005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24284e225c34199809b213bae06e3a4', 'request_id': 'req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.213675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c063e1aba4aa4187b3e6166a9ac793f0', 'request_id': 'req-5216022b-7b7f-4276-8d62-3b988ca4f81a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.234869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c063e1aba4aa4187b3e6166a9ac793f0', 'request_id': 'req-5216022b-7b7f-4276-8d62-3b988ca4f81a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.241186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] The token cache is being invalidate because project a24284e225c34199809b213bae06e3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:16.241662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.242657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ac34028-b9bd-4f4d-a4f9-af2e5964afda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.248698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 996/1987] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:16 2022] DELETE /identity/v3/projects/a24284e225c34199809b213bae06e3a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:16.251348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7723cb-b7cb-47af-8498-b08a6cc44703 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.251735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7723cb-b7cb-47af-8498-b08a6cc44703 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.255791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7723cb-b7cb-47af-8498-b08a6cc44703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.261572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] The token cache is being invalidate because project c063e1aba4aa4187b3e6166a9ac793f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:16.261865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.264227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.267783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 992/1988] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:16 2022] DELETE /identity/v3/projects/c063e1aba4aa4187b3e6166a9ac793f0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:16.274966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.277570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.281517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e7723cb-b7cb-47af-8498-b08a6cc44703 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:16.282784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e7723cb-b7cb-47af-8498-b08a6cc44703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.366075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216022b-7b7f-4276-8d62-3b988ca4f81a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.367642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 997/1989] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:16.369802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.370221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.370642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.372745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.373442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.373723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.373997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] PATH_INFO: `/v3/users/c19d0f1cd5e3420aa1ddf3fe3deaffd4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.375207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19d0f1cd5e3420aa1ddf3fe3deaffd4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.379058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.421796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:16.423313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.432048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 998/1990] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:16.436487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.446374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d8a83-cab9-44db-9037-bf261d6a2d73 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.451597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.452214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.452887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.452887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.453693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:16.453992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.455684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.457105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19d0f1cd5e3420aa1ddf3fe3deaffd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.470825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19d0f1cd5e3420aa1ddf3fe3deaffd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.471927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19d0f1cd5e3420aa1ddf3fe3deaffd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.472732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 999/1991] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:38:16.477174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.488398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05acc44-2bef-46ea-9b01-26539f7f14c4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.490338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19d0f1cd5e3420aa1ddf3fe3deaffd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.497057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.498123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19d0f1cd5e3420aa1ddf3fe3deaffd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:16.498297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.498763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.499308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.500210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.502412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.509444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 993/1992] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:16 2022] DELETE /identity/v3/users/c19d0f1cd5e3420aa1ddf3fe3deaffd4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:16.543021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1000/1993] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:16 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:16.544347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.545901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.594643 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:16.619366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0077a-a847-48f0-a834-0a597bdf121a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.680858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.681495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.681770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.682036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.682711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.684810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.710077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 994/1994] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:16 2022] POST /identity/v3/users => generated 280 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:16.711424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.722509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f56c75-3635-4364-b855-148b349d7706 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.728247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.729066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.729413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.729752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.730534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.732363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.745420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1001/1995] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:16.748854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.779125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35675d2-33cc-410b-8149-cdcb7c180d99 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.785635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b0da5-84f8-400d-9eb9-e2fa7f811a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.786271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0da5-84f8-400d-9eb9-e2fa7f811a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.786545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0da5-84f8-400d-9eb9-e2fa7f811a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.786814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0da5-84f8-400d-9eb9-e2fa7f811a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.787509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b0da5-84f8-400d-9eb9-e2fa7f811a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.789187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b0da5-84f8-400d-9eb9-e2fa7f811a97 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.803523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 995/1996] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:16.804700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.816247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14dad1-4984-4b13-9b38-54f4f297e63c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:16.821564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:16.822246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.822518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.822790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] PATH_INFO: `/v3/projects/a6cd0ce800e6487c8a1fcdbd500f06ed/users/9f26fc4b9cac4d32a1c088b26c2db96b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.839711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6cd0ce800e6487c8a1fcdbd500f06ed, user_id=9f26fc4b9cac4d32a1c088b26c2db96b, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:16.842906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:16.868096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1002/1997] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:16 2022] PUT /identity/v3/projects/a6cd0ce800e6487c8a1fcdbd500f06ed/users/9f26fc4b9cac4d32a1c088b26c2db96b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:16.883287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:16.884094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:16.884657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:16.907957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:16.910273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:16.919469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] MFA Rules not processed for user `9f26fc4b9cac4d32a1c088b26c2db96b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:16.921065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.006565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4179e3-84bc-4926-b3a9-5fdbcbc8495a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.012818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.013580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.013826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.014093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] PATH_INFO: `/v3/projects/46e947d99f68495aa555eafe45cbb6f2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.017038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e947d99f68495aa555eafe45cbb6f2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.019626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.026967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:17.065665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 996/1998] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:16 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:17.094583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e947d99f68495aa555eafe45cbb6f2', 'request_id': 'req-53b61a31-4afd-4720-8beb-99cb5bf13d7d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.113638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e947d99f68495aa555eafe45cbb6f2', 'request_id': 'req-53b61a31-4afd-4720-8beb-99cb5bf13d7d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.114150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.115854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.121269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e947d99f68495aa555eafe45cbb6f2', 'request_id': 'req-53b61a31-4afd-4720-8beb-99cb5bf13d7d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.142903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] The token cache is being invalidate because project 46e947d99f68495aa555eafe45cbb6f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:17.142903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.143572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-53b61a31-4afd-4720-8beb-99cb5bf13d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.150413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1003/1999] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:16 2022] DELETE /identity/v3/projects/46e947d99f68495aa555eafe45cbb6f2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:17.152005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647e229-d0dd-4f2a-a85d-34102eb9540e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.152289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647e229-d0dd-4f2a-a85d-34102eb9540e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.152543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647e229-d0dd-4f2a-a85d-34102eb9540e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.193115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7647e229-d0dd-4f2a-a85d-34102eb9540e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:17.194350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7647e229-d0dd-4f2a-a85d-34102eb9540e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.208238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ff39d-80d8-4cc5-bb20-802b871b49b2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.233552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:17.289402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1004/2000] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:17.291061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.291770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.292117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.292531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.294549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.298004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.298438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.299905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.320391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:17.321632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.330701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1005/2001] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:17.335301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.345251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9f1b-fef0-4829-a10c-67783ef96007 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.350224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.350822 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.351102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.351323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.352048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:17.352426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.354573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.367181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1006/2002] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:38:17.368604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.372250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.372385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.383243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73b11-1d9b-4df1-9b54-056254c87cc6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.383822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 997/2003] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:17 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:17.388460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.389338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.389535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.389766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.390430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.392457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.400367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.404074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.419478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f0e2d-5a56-40a2-bc9b-46d4f0bf7cc7 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.424732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.425418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.425691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.426098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] PATH_INFO: `/v3/users/9f26fc4b9cac4d32a1c088b26c2db96b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.427635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f26fc4b9cac4d32a1c088b26c2db96b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.430894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.433658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1007/2004] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:17.439317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.504262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:17.508870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f26fc4b9cac4d32a1c088b26c2db96b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.519446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb6fc1-6f6f-4038-8880-d88295c93046 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.519656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f26fc4b9cac4d32a1c088b26c2db96b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.520203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f26fc4b9cac4d32a1c088b26c2db96b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.527668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.528670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.529154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.529597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.531146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.533568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.552255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f26fc4b9cac4d32a1c088b26c2db96b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.561973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-537c49ad-a6af-4e0d-a3f4-3277b3fc17ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f26fc4b9cac4d32a1c088b26c2db96b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:17.573888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1008/2005] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:17.577445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 998/2006] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:17 2022] DELETE /identity/v3/users/9f26fc4b9cac4d32a1c088b26c2db96b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:17.578131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.581711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.582167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.582311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.582489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.595065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca1649-3a8a-4102-b519-09c61e65979c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.602049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:17.603453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.650830 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:17.662004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.663063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.663458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.664068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.665023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.668474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.683976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1009/2007] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:17.690273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.704401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821827e-3b1a-4838-9ed1-6032c61c913c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.705220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 999/2008] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:17.713195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.713925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.714265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.714597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.714982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.715782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.717483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.717587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.737538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1010/2009] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:17.739154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:17.752650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e59e-6c99-4da6-a6f2-80f4e3f47578 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.764706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33335b-a362-490d-b258-8b70f890606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.765453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33335b-a362-490d-b258-8b70f890606d admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.765726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33335b-a362-490d-b258-8b70f890606d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.766271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33335b-a362-490d-b258-8b70f890606d admin admin] PATH_INFO: `/v3/projects/c31350408edf40508d88905ff6d74b34/users/32d872d0d12e4c31aaa05d41cc1b5ebf/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.780291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc6979-cf99-4656-a2c2-5215b3ea4b67 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:17.784517 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:17.784946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.785224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.785494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.785837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33335b-a362-490d-b258-8b70f890606d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31350408edf40508d88905ff6d74b34, user_id=32d872d0d12e4c31aaa05d41cc1b5ebf, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.788913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33335b-a362-490d-b258-8b70f890606d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.795627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:17.797530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:17.807901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1011/2010] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:17 2022] PUT /identity/v3/projects/c31350408edf40508d88905ff6d74b34/users/32d872d0d12e4c31aaa05d41cc1b5ebf/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:17.809154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1000/2011] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:17.823052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:17.823427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:17.823608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:17.857753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] MFA Rules not processed for user `32d872d0d12e4c31aaa05d41cc1b5ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:17.859545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:17.967140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1012/2012] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:17.973639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.024872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465af122-ff03-4f2d-869b-084acfcdeab5 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.154867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.155762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.156047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.156316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.157045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.159071 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.200892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1001/2013] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:17 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:18.202071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.219751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453ff7d-a733-4313-b5c2-bbc353b913f7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.254672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.256732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:18.257347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:18.311652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.312339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.312615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.312920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.313588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.315509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.330248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e67e4-5cbf-4683-9c84-af05ca5b3c31 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.336474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.337167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.337444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.337714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] PATH_INFO: `/v3/projects/a6cd0ce800e6487c8a1fcdbd500f06ed` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.338981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6cd0ce800e6487c8a1fcdbd500f06ed)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.341719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1013/2014] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:18.341843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.345765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.354384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:18.357286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b916-d47d-489d-9d74-73276b3a693b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.362491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.363125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.363405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.363681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.364513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.366428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.379389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1014/2015] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:18.384399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.394715 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4cfe1-6e2b-4c7e-8198-49377c00c889 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.402888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6cd0ce800e6487c8a1fcdbd500f06ed', 'request_id': 'req-d70829f4-28c9-46b6-8d4e-d04a335f440e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:18.411449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6cd0ce800e6487c8a1fcdbd500f06ed', 'request_id': 'req-d70829f4-28c9-46b6-8d4e-d04a335f440e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:18.418796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6cd0ce800e6487c8a1fcdbd500f06ed', 'request_id': 'req-d70829f4-28c9-46b6-8d4e-d04a335f440e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:18.433125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] The token cache is being invalidate because project a6cd0ce800e6487c8a1fcdbd500f06ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:18.433432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:18.434125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d70829f4-28c9-46b6-8d4e-d04a335f440e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:18.435797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1002/2016] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:18 2022] DELETE /identity/v3/projects/a6cd0ce800e6487c8a1fcdbd500f06ed => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:18.447034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.447380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.447664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.468327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:18.469454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358e16-fddb-4faa-ba79-2bd55bc18fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.469652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:18.470094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358e16-fddb-4faa-ba79-2bd55bc18fde admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.470374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358e16-fddb-4faa-ba79-2bd55bc18fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.470659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358e16-fddb-4faa-ba79-2bd55bc18fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.471375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08358e16-fddb-4faa-ba79-2bd55bc18fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.473180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08358e16-fddb-4faa-ba79-2bd55bc18fde admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.478449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1003/2017] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:18.484187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1015/2018] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:18.488490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.490316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.490719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.490998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.491767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:18.504235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d3b12-7a50-4a6c-8c5a-8464be11f0b8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.509472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20289719-0cd4-4dd5-8ae2-cfa15a2e612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.510158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289719-0cd4-4dd5-8ae2-cfa15a2e612d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.510437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289719-0cd4-4dd5-8ae2-cfa15a2e612d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.510708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20289719-0cd4-4dd5-8ae2-cfa15a2e612d admin admin] PATH_INFO: `/v3/projects/05c3d7dc7f02404a971320befa1d8860/users/aef125f4d30c430db7715cb66097021c/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.512498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:18.513757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:18.522090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1016/2019] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:18.527877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20289719-0cd4-4dd5-8ae2-cfa15a2e612d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c3d7dc7f02404a971320befa1d8860, user_id=aef125f4d30c430db7715cb66097021c, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.533273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.533472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20289719-0cd4-4dd5-8ae2-cfa15a2e612d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.543456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b1d35-7697-4ed3-9655-86c4481826cc None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.549295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1004/2020] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:18 2022] PUT /identity/v3/projects/05c3d7dc7f02404a971320befa1d8860/users/aef125f4d30c430db7715cb66097021c/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:18.631727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.632728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.633282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.633708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.634540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:18.635142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.637462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.647675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.647963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.648234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.657995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1017/2021] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:18.659128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.670255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec91de-39a5-41c6-85aa-fb3dd4b2eabe admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.675618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.676856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.677409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.677699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.678430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.680866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.681268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] MFA Rules not processed for user `aef125f4d30c430db7715cb66097021c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:18.682659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:18.716576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1018/2022] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:18.719359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.733289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:18.760504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1005/2023] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:18.765173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.776181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23d0eb-cd7e-4a60-8405-40a15c6471d7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.790845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7616be3-6d9f-4198-be65-8ff84007addb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.795733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.796396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.796711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.796946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.797615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.799555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.822023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1019/2024] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:18.826375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.833471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:18.836576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf60d4-5a90-4a43-85f7-cf97668462fb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.840491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.841344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.841658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.842170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.842278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.842551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.842838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.843231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.843539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.844287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.845243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.847216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.856412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1020/2025] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:18.860363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.870446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205715e-0387-4312-884d-12796edba8b6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.875393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.876013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.876401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.876675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.877400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.878628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1006/2026] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:18.882322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.883201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.898707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1021/2027] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:18.902646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.914676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5cd97-d64a-4dfd-8ceb-8558ddd0d64e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.954059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a167552-a41f-43f2-8bbc-48cda2e15487 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:18.958969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.959620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.959900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.960172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:18.961080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:18.963191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:18.989455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1007/2028] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:18 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:18.994232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:18.997236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c60fb-7020-4271-a953-93e06f49f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:18.997949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c60fb-7020-4271-a953-93e06f49f6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:18.998228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c60fb-7020-4271-a953-93e06f49f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:18.998500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c60fb-7020-4271-a953-93e06f49f6f4 admin admin] PATH_INFO: `/v3/projects/b0dbc7429a904dccb5cf657022e95948/users/c21d975580e248c5b4c208cd6826ba61/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.005535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12138f-fc78-4708-9b56-2778eb89b6a0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.011062 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.011729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.012011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.012287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.013064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.014686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.015365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c60fb-7020-4271-a953-93e06f49f6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0dbc7429a904dccb5cf657022e95948, user_id=c21d975580e248c5b4c208cd6826ba61, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.018257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c60fb-7020-4271-a953-93e06f49f6f4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.027961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1008/2029] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:19.032103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.036385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1022/2030] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:18 2022] PUT /identity/v3/projects/b0dbc7429a904dccb5cf657022e95948/users/c21d975580e248c5b4c208cd6826ba61/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:19.043392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b1eae-54b1-4313-b4c8-89491601e206 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.046700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.046988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.047264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.079573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] MFA Rules not processed for user `c21d975580e248c5b4c208cd6826ba61`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:19.081001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.120731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.121698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.122046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.122314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] PATH_INFO: `/v3/projects/19ea58dc7ee443d6911202e53bee5ad7/users/0f5313dabb16497fb144f4cf395b34f2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.138856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ea58dc7ee443d6911202e53bee5ad7, user_id=0f5313dabb16497fb144f4cf395b34f2, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.140514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:19.141652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.155672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1009/2031] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:19 2022] PUT /identity/v3/projects/19ea58dc7ee443d6911202e53bee5ad7/users/0f5313dabb16497fb144f4cf395b34f2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:19.159655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.163693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1023/2032] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:19.175654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.177096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.210286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:19.234458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5188f-365a-4733-a238-dcbfa51f53de admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.239927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.240690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.241008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.241291 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.242151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.243769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.244991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a922b-bbaf-4edb-be96-9d49aa87b4d5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.256485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1010/2033] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:19.260633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.270210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9e49a-025b-490c-98ae-7ba5e30893d0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.275065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cc958-07ea-4fbd-92cf-632a5f88883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.275719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cc958-07ea-4fbd-92cf-632a5f88883a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.275993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cc958-07ea-4fbd-92cf-632a5f88883a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.276255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cc958-07ea-4fbd-92cf-632a5f88883a admin admin] PATH_INFO: `/v3/projects/19ea58dc7ee443d6911202e53bee5ad7/users/0f5313dabb16497fb144f4cf395b34f2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.278381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cc958-07ea-4fbd-92cf-632a5f88883a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ea58dc7ee443d6911202e53bee5ad7, user_id=0f5313dabb16497fb144f4cf395b34f2, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.281484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cc958-07ea-4fbd-92cf-632a5f88883a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.292205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1011/2034] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:19 2022] PUT /identity/v3/projects/19ea58dc7ee443d6911202e53bee5ad7/users/0f5313dabb16497fb144f4cf395b34f2/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:19.301017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.301017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.301017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.301017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.307316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.308113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.308399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.348504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] MFA Rules not processed for user `0f5313dabb16497fb144f4cf395b34f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:19.349758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.367629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.369630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.381260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1024/2035] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:19.416963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.425749 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33854674-9086-4c04-8006-de2c44fa84f3 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.436806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1012/2036] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:19.445980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.447461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.483575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.484251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.484556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.484865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] PATH_INFO: `/v3/users/c21d975580e248c5b4c208cd6826ba61` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.486059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21d975580e248c5b4c208cd6826ba61)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.488024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.490617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:19.504970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc190-155a-41ad-81de-b9c7b880124d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.509466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.510105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.510657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.510657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.521778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.523246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.533106 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1013/2037] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:19.536329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21d975580e248c5b4c208cd6826ba61'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:19.546252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21d975580e248c5b4c208cd6826ba61'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:19.546965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21d975580e248c5b4c208cd6826ba61'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:19.556873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21d975580e248c5b4c208cd6826ba61'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:19.557699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d2a50309-7b28-42b3-a638-f58d855880a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21d975580e248c5b4c208cd6826ba61'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:19.581787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.589107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1025/2038] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:19 2022] DELETE /identity/v3/users/c21d975580e248c5b4c208cd6826ba61 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:19.590446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.590732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.591005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.591653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a388-a538-49aa-94c6-5abfed5580b4 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.618268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:19.619859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.645301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.646075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.646355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.646650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.648958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.661366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:19.696914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1026/2039] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:19.707536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:19.709255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:19.732293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.734136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.745523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1014/2040] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:19.793801 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b2698-f8f8-4061-84a5-c12d4c087032 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:19.797504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:19.798129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:19.798404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:19.798675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:19.808608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:19.810180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:19.819958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1027/2041] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:20.135694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:20.137159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:20.151828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1d332-9fe4-40dd-8e47-6ed1091a12fe service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:20.158081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:20.158831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:20.159190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:20.159472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] PATH_INFO: `/v3/projects/b0dbc7429a904dccb5cf657022e95948` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:20.160805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0dbc7429a904dccb5cf657022e95948)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:20.163358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:20.221138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0dbc7429a904dccb5cf657022e95948', 'request_id': 'req-1992777d-156e-46b0-a168-baaf22924840'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:20.228210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0dbc7429a904dccb5cf657022e95948', 'request_id': 'req-1992777d-156e-46b0-a168-baaf22924840'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:20.236700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0dbc7429a904dccb5cf657022e95948', 'request_id': 'req-1992777d-156e-46b0-a168-baaf22924840'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:20.250674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] The token cache is being invalidate because project b0dbc7429a904dccb5cf657022e95948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:20.250964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:20.251580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1992777d-156e-46b0-a168-baaf22924840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:20.253532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1015/2042] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:20 2022] DELETE /identity/v3/projects/b0dbc7429a904dccb5cf657022e95948 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:22.693979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:22.695418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:22.708382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538d31f-07de-432f-96bb-5610a45bf37f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:22.731270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:22.780900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:22.781550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:22.781821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:22.782084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] PATH_INFO: `/v3/users/32d872d0d12e4c31aaa05d41cc1b5ebf` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:22.783230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d872d0d12e4c31aaa05d41cc1b5ebf)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:22.785213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:22.832362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d872d0d12e4c31aaa05d41cc1b5ebf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:22.840810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d872d0d12e4c31aaa05d41cc1b5ebf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:22.841473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d872d0d12e4c31aaa05d41cc1b5ebf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:22.848352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d872d0d12e4c31aaa05d41cc1b5ebf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:22.848999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d872d0d12e4c31aaa05d41cc1b5ebf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:22.867360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1028/2043] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:22 2022] DELETE /identity/v3/users/32d872d0d12e4c31aaa05d41cc1b5ebf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:22.871899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:22.872370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:22.872476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:22.892228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:22.893470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:22.956338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1016/2044] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:22.966161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:22.967626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:23.027289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502b02-85d5-41b0-8cf2-2d35a3a4159e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:23.030880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:23.031501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:23.031780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:23.032052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:23.041242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:23.042675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:23.062028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1029/2045] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:23.344729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:23.346639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:23.370340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b90f4-9a6b-426f-b0e0-1e480282d91f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:23.376132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:23.376739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:23.377102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:23.377369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] PATH_INFO: `/v3/projects/c31350408edf40508d88905ff6d74b34` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:23.378639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31350408edf40508d88905ff6d74b34)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:23.381012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:23.416271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:23.433947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31350408edf40508d88905ff6d74b34', 'request_id': 'req-fffd032b-e60d-4ae2-803f-b186357a9786'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.440871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31350408edf40508d88905ff6d74b34', 'request_id': 'req-fffd032b-e60d-4ae2-803f-b186357a9786'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.448118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31350408edf40508d88905ff6d74b34', 'request_id': 'req-fffd032b-e60d-4ae2-803f-b186357a9786'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.461128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] The token cache is being invalidate because project c31350408edf40508d88905ff6d74b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:23.461413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.462015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.463598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1017/2046] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:23 2022] DELETE /identity/v3/projects/c31350408edf40508d88905ff6d74b34 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:23.467946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:23.477651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1f85-04c8-4b44-b482-b6337ea5589a service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:23.528352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:23.545072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:23.545725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:23.546002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:23.546303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] PATH_INFO: `/v3/users/aef125f4d30c430db7715cb66097021c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:23.547521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef125f4d30c430db7715cb66097021c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:23.549656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:23.611043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef125f4d30c430db7715cb66097021c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.620314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef125f4d30c430db7715cb66097021c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.621137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef125f4d30c430db7715cb66097021c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.629557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef125f4d30c430db7715cb66097021c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.630240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef125f4d30c430db7715cb66097021c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:23.650246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1030/2047] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:23 2022] DELETE /identity/v3/users/aef125f4d30c430db7715cb66097021c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:24.401028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:24.402597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:24.417511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd032b-e60d-4ae2-803f-b186357a9786 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:24.486470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:24.487488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:24.487951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:24.488440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] PATH_INFO: `/v3/projects/05c3d7dc7f02404a971320befa1d8860` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:24.490169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c3d7dc7f02404a971320befa1d8860)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:24.492633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:24.546209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c3d7dc7f02404a971320befa1d8860', 'request_id': 'req-6d77455b-3ce6-4e23-b3fa-800296108de8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.556916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c3d7dc7f02404a971320befa1d8860', 'request_id': 'req-6d77455b-3ce6-4e23-b3fa-800296108de8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.563037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c3d7dc7f02404a971320befa1d8860', 'request_id': 'req-6d77455b-3ce6-4e23-b3fa-800296108de8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.577305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] The token cache is being invalidate because project 05c3d7dc7f02404a971320befa1d8860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:24.577595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.578202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.579743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1018/2048] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:24 2022] DELETE /identity/v3/projects/05c3d7dc7f02404a971320befa1d8860 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:24.584151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:24.585601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:24.598738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d4b19-ba14-4e3e-9e85-8810e08d1327 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:24.660008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:24.660563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:24.660874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:24.661413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] PATH_INFO: `/v3/users/0f5313dabb16497fb144f4cf395b34f2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:24.662315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5313dabb16497fb144f4cf395b34f2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:24.664253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:24.710492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5313dabb16497fb144f4cf395b34f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.718158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5313dabb16497fb144f4cf395b34f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.718761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5313dabb16497fb144f4cf395b34f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.725689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5313dabb16497fb144f4cf395b34f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.726302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-97fe0c3f-c000-4f28-a9bb-5dd5ad04708d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5313dabb16497fb144f4cf395b34f2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:24.732541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:24.750988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1031/2049] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:24 2022] DELETE /identity/v3/users/0f5313dabb16497fb144f4cf395b34f2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:25.252607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.254435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:25.279319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77455b-3ce6-4e23-b3fa-800296108de8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.336915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:25.358205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.358879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.359377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.359732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] PATH_INFO: `/v3/projects/19ea58dc7ee443d6911202e53bee5ad7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.361505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ea58dc7ee443d6911202e53bee5ad7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.364798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.434986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ea58dc7ee443d6911202e53bee5ad7', 'request_id': 'req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:25.441931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ea58dc7ee443d6911202e53bee5ad7', 'request_id': 'req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:25.448534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ea58dc7ee443d6911202e53bee5ad7', 'request_id': 'req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:25.462309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] The token cache is being invalidate because project 19ea58dc7ee443d6911202e53bee5ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:25.462593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:25.463197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b6add52b-e3ef-4ae8-ab0f-5bdce69380be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:25.465370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1019/2050] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:25 2022] DELETE /identity/v3/projects/19ea58dc7ee443d6911202e53bee5ad7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:25.471610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.471889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.472181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.497499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:25.498908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:25.574514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1032/2051] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:25.584181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.584525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.584913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.606866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:25.608117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:25.620024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1020/2052] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:25.621304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.634900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afdb78-5a82-4a2b-9838-82326afbcfb6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.641229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.641934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.642216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.642482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.643304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:25.643770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.645935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.658760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1033/2053] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:25.663530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.673943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2b648-d4eb-4789-8210-d3d73dbce9c4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.678970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.679611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.680050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.680319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.681167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.683737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.714402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1021/2054] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:25 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:25.718835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.729540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca62c5-0a13-44b6-bb42-858285721fe7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.795697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.795697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.796032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.796486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.797479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.800493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.839029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1034/2055] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:25 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:25.840199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.855033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d08bb-6c00-4d92-bb3b-9ecf5fb2c2ff admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.860386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56369a-3440-402c-904d-ea55414b7851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.861610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56369a-3440-402c-904d-ea55414b7851 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.861965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56369a-3440-402c-904d-ea55414b7851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.862316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56369a-3440-402c-904d-ea55414b7851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.863098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56369a-3440-402c-904d-ea55414b7851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.864806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56369a-3440-402c-904d-ea55414b7851 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.876086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1022/2056] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:25.880463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.891226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5ba57-c719-4673-b5f0-3c65aebdcc0b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.897071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a876741-4ffe-40a0-9660-1e7fee04ed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.897699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a876741-4ffe-40a0-9660-1e7fee04ed3e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.897976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a876741-4ffe-40a0-9660-1e7fee04ed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.898246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a876741-4ffe-40a0-9660-1e7fee04ed3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.898945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a876741-4ffe-40a0-9660-1e7fee04ed3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.900563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a876741-4ffe-40a0-9660-1e7fee04ed3e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.908522 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53a00-18c2-40e0-8624-be6669e2dcaa None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.909557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53a00-18c2-40e0-8624-be6669e2dcaa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.910083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53a00-18c2-40e0-8624-be6669e2dcaa None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.912164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1023/2057] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:38:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:38:25.912591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1035/2058] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:25.916424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53a00-18c2-40e0-8624-be6669e2dcaa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:25.929395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53a00-18c2-40e0-8624-be6669e2dcaa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:25.934039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9322866-1afa-41cc-b31b-c258df3bcc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:25.934720 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322866-1afa-41cc-b31b-c258df3bcc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.934995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322866-1afa-41cc-b31b-c258df3bcc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.935267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9322866-1afa-41cc-b31b-c258df3bcc72 admin admin] PATH_INFO: `/v3/projects/2ccef57782ae4a38ab22206aa9289b23/users/4814673c66784324987282c700e9bde2/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.944346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.944696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.945705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:25.950890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322866-1afa-41cc-b31b-c258df3bcc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccef57782ae4a38ab22206aa9289b23, user_id=4814673c66784324987282c700e9bde2, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:25.953703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9322866-1afa-41cc-b31b-c258df3bcc72 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:25.972939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] MFA Rules not processed for user `5686862728f34d64af248e12358541d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:25.974215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:25.974542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1024/2059] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:25 2022] PUT /identity/v3/projects/2ccef57782ae4a38ab22206aa9289b23/users/4814673c66784324987282c700e9bde2/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:25.987936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:25.989116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:25.989563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.026980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] MFA Rules not processed for user `4814673c66784324987282c700e9bde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:26.028259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.030835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:26.050460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1036/2060] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:38:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:26.051300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.052596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.094366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1025/2061] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:26.099545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.100989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.110288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:26.116965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c9435-818e-4f92-8cee-749c15a631b7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.117107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0e7ce-089f-4074-9d9f-f32e73426b68 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.121865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.122516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.122793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.122995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.123122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] PATH_INFO: `/v3/users/e518e5691a7247de9d04ae9828809498` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.123608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.123989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.124256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.124403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e518e5691a7247de9d04ae9828809498)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.126357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.133259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.135030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.144897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1026/2062] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:26.145765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.147112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.177697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e518e5691a7247de9d04ae9828809498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.186908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e518e5691a7247de9d04ae9828809498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.187531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e518e5691a7247de9d04ae9828809498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.194710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e518e5691a7247de9d04ae9828809498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.195482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e518e5691a7247de9d04ae9828809498'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.206464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ec3b-5980-46dc-b8a9-1b3bc6f70f74 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.210105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.210722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.210989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.211252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.212827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1037/2063] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:26 2022] DELETE /identity/v3/users/e518e5691a7247de9d04ae9828809498 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:26.213016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.224996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.226542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.236272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1027/2064] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:26.417734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.419187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.475468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.477009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.510462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ab43-b5b1-464b-a8ef-9c4a7082419b service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.523014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70749cf0-85e3-44ac-9e71-4f95c68a8ae8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.528279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.528974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.529257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.529531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] PATH_INFO: `/v3/projects/a2eb8b8a120e4e87b4407d0abb3b3f07` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.530738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2eb8b8a120e4e87b4407d0abb3b3f07)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.532649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.541064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:26.587614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.589795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.590075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.590347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] PATH_INFO: `/v3/users/5a15faa7e83d4feaa69f62a78783e4cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.592115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a15faa7e83d4feaa69f62a78783e4cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.595167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.595615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2eb8b8a120e4e87b4407d0abb3b3f07', 'request_id': 'req-3a6817cc-4572-4729-af85-6284b5cca03e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.604621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2eb8b8a120e4e87b4407d0abb3b3f07', 'request_id': 'req-3a6817cc-4572-4729-af85-6284b5cca03e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.617250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2eb8b8a120e4e87b4407d0abb3b3f07', 'request_id': 'req-3a6817cc-4572-4729-af85-6284b5cca03e'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.635866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] The token cache is being invalidate because project a2eb8b8a120e4e87b4407d0abb3b3f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:26.636456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.637086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3a6817cc-4572-4729-af85-6284b5cca03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.638876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1038/2065] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:26 2022] DELETE /identity/v3/projects/a2eb8b8a120e4e87b4407d0abb3b3f07 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:26.650358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ccab5-5083-48d6-b6d0-fa677781f2ad None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.650789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ccab5-5083-48d6-b6d0-fa677781f2ad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.651100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ccab5-5083-48d6-b6d0-fa677781f2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.652995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a15faa7e83d4feaa69f62a78783e4cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.661867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a15faa7e83d4feaa69f62a78783e4cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.662868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a15faa7e83d4feaa69f62a78783e4cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.679093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a15faa7e83d4feaa69f62a78783e4cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.683662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6a3ccab5-5083-48d6-b6d0-fa677781f2ad None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:26.684962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6a3ccab5-5083-48d6-b6d0-fa677781f2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.686973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7b5b37b6-d2c6-432a-9150-01c4b866ffc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a15faa7e83d4feaa69f62a78783e4cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:26.693288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1028/2066] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:26 2022] DELETE /identity/v3/users/5a15faa7e83d4feaa69f62a78783e4cc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:26.698098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.698378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.698651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.717404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:26.718593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.723584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:26.753297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1039/2067] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:26.762490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.762763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.763035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.782618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1029/2068] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:26.782789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:26.784114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.792427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.793347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1040/2069] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:26.793817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:26.797462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.807196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196e6fc-ee9e-4168-b8ab-2acbf1e7e57b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.812310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.812939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.813213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.813476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.814158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:26.814452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.816019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.827911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1041/2070] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:38:26.832157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.841468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40e304-d41c-4a0c-80c9-e37cacc69304 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.845982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.846602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.846839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.847102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.847743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.849638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.850332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd1f9-db90-4b45-bd30-2cad94d91c9b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.854561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.855511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.855786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.856055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.866654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.868107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.878022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1042/2071] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:26 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:26.880139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1030/2072] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:26.882643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.940641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:26.945592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba262329-e847-448c-bbe1-513fa0f1a870 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.951319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.952238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.952865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.953215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.953871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:26.955791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:26.977907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1043/2073] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:26 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:26.982009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:26.992989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745c4c9-2f3e-41d8-ba8f-3cca6f4a42da service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:26.997681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:26.998302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:26.998572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:26.998921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:26.999591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.001221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.012002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1031/2074] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:27.016017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.031662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7314379-d33f-40b4-808e-28fbeb5ac7eb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.036870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfb943-ddc0-4d9d-8dae-da066b5f2ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.037507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb943-ddc0-4d9d-8dae-da066b5f2ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.037783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb943-ddc0-4d9d-8dae-da066b5f2ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.038054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb943-ddc0-4d9d-8dae-da066b5f2ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.038799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfb943-ddc0-4d9d-8dae-da066b5f2ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.040428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfb943-ddc0-4d9d-8dae-da066b5f2ee2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.056184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.056713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1044/2075] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:27.066912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d071-a0b5-4f65-940d-383f7af90d09 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.071550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.072351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.072632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.073196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] PATH_INFO: `/v3/projects/45434f8a513f4c53925da7a110c6d6f9/users/42a7b55e562940a687ecee9495854985/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.088684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45434f8a513f4c53925da7a110c6d6f9, user_id=42a7b55e562940a687ecee9495854985, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.091147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.104861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1032/2076] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:27 2022] PUT /identity/v3/projects/45434f8a513f4c53925da7a110c6d6f9/users/42a7b55e562940a687ecee9495854985/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:27.115416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.115700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.115997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.146043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] MFA Rules not processed for user `42a7b55e562940a687ecee9495854985`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:27.147408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.168981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.170760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.232454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1045/2077] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:27.237115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.254535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a995fde-6055-4024-b8c6-69210c2949c5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.259137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.259778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.260048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.260601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] PATH_INFO: `/v3/projects/8342abc20bb04467840b9e4f8565e8d9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.261795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8342abc20bb04467840b9e4f8565e8d9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.263707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.286158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:27.299255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f222e9-41f9-42b9-9e15-9ac50251c930 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.311439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8342abc20bb04467840b9e4f8565e8d9', 'request_id': 'req-ca770761-a473-4cf0-9942-fd934f4d7906'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:27.317483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8342abc20bb04467840b9e4f8565e8d9', 'request_id': 'req-ca770761-a473-4cf0-9942-fd934f4d7906'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:27.323174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8342abc20bb04467840b9e4f8565e8d9', 'request_id': 'req-ca770761-a473-4cf0-9942-fd934f4d7906'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:27.337285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] The token cache is being invalidate because project 8342abc20bb04467840b9e4f8565e8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:27.337668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:27.338322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ca770761-a473-4cf0-9942-fd934f4d7906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:27.339901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1033/2078] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:27 2022] DELETE /identity/v3/projects/8342abc20bb04467840b9e4f8565e8d9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:27.345427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a788-cc31-4ec0-b954-62f9b53a11d4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.345828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a788-cc31-4ec0-b954-62f9b53a11d4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.346104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a788-cc31-4ec0-b954-62f9b53a11d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.353565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.354184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.354459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.354761 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.355388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.357353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.365732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0710a788-cc31-4ec0-b954-62f9b53a11d4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:27.366935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0710a788-cc31-4ec0-b954-62f9b53a11d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.373062 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:27.374999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1034/2079] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:27.383638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.383949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.384224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.390108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1046/2080] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/projects => generated 361 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:27.391448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.393374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.404238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:27.405453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.459857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943c69c-5cdc-4fa2-954f-87ca2eaea706 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.464825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.465198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1035/2081] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:38:27.465450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.465753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.466026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.466717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.469089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.469388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.478161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bcbe6-72e5-4f77-8122-d7dfb4fbfbe9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.483506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.484125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.484426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.484713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.485445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:27.485741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.487366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.494936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1047/2082] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:27.496149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.507209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19661d2b-9f6a-44b1-9f27-5b3d24d9083e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.508916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1036/2083] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:38:27.510180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.513173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.513829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.514088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.514469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.515466 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.517334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.521516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417987d-f652-4b4a-9fb4-468b4641d5e2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.526738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.527366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.527630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.527947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1048/2084] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:27.529134 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.529862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.531836 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.532039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.542057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed320-4d99-4308-a3d1-caabd5c4f6f3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.547841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.548412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.548841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.549115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] PATH_INFO: `/v3/projects/f3cf7034871444938deed61c83713f02/users/77ef7235b6074df78b78bd786b1f5cdc/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.561585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1037/2085] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:27.565634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.571385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cf7034871444938deed61c83713f02, user_id=77ef7235b6074df78b78bd786b1f5cdc, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.573843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.590316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1049/2086] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:27 2022] PUT /identity/v3/projects/f3cf7034871444938deed61c83713f02/users/77ef7235b6074df78b78bd786b1f5cdc/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:27.594130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.641378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415e78c-f31f-454a-b745-7f98adac83d7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.657818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:27.662177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe82b02-0171-41e5-90c4-0117e805163e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.674419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.675128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.675474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.676340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.677343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.680508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.701354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1050/2087] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:27.703104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.711993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.712469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.712793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.713067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.713728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.715641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.722790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e22ef0-261a-496a-b612-9d0e91837046 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.727873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e3345-296e-4ff0-b186-5bded3c257bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.728686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e3345-296e-4ff0-b186-5bded3c257bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.728986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e3345-296e-4ff0-b186-5bded3c257bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.729323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e3345-296e-4ff0-b186-5bded3c257bb admin admin] PATH_INFO: `/v3/projects/f3cf7034871444938deed61c83713f02/users/77ef7235b6074df78b78bd786b1f5cdc/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.731517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e3345-296e-4ff0-b186-5bded3c257bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cf7034871444938deed61c83713f02, user_id=77ef7235b6074df78b78bd786b1f5cdc, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.734636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e3345-296e-4ff0-b186-5bded3c257bb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.743451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1038/2088] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:27.747340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.747650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1051/2089] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:27 2022] PUT /identity/v3/projects/f3cf7034871444938deed61c83713f02/users/77ef7235b6074df78b78bd786b1f5cdc/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:27.759324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.759609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.759908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.760572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fea388-b0f3-4930-a560-1bba92acfcae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.805866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] MFA Rules not processed for user `77ef7235b6074df78b78bd786b1f5cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:27.807373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.824946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.825620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.825878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.826177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.826914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.828569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.830154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:27.844650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1039/2090] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:27.848637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.859179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3f465-23e8-4590-b0d8-b5140e061380 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.863984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.864618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.864937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.865245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.865985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.867565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.882458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1040/2091] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:27.884197 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.887779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1052/2092] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:27.900861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:27.901890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed4b55-fb26-43bf-89bd-ee167502140d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.902662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:27.907144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d227776-2584-4649-98d2-52cab13abd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:27.907972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d227776-2584-4649-98d2-52cab13abd38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.908309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d227776-2584-4649-98d2-52cab13abd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.908692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d227776-2584-4649-98d2-52cab13abd38 admin admin] PATH_INFO: `/v3/projects/1716a51c2dda40c498c5f501fe89cc44/users/1bb74653b8784120859dc95daa62c66b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.924659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d227776-2584-4649-98d2-52cab13abd38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1716a51c2dda40c498c5f501fe89cc44, user_id=1bb74653b8784120859dc95daa62c66b, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:27.927490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d227776-2584-4649-98d2-52cab13abd38 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:27.944804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1041/2093] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:27 2022] PUT /identity/v3/projects/1716a51c2dda40c498c5f501fe89cc44/users/1bb74653b8784120859dc95daa62c66b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:27.955019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:27.955374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:27.955714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:27.973297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236f48f-6ce5-46f7-a64c-f11c2e8ca8d4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:27.988684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] MFA Rules not processed for user `1bb74653b8784120859dc95daa62c66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:27.990756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:28.029313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:28.029986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:28.030269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:28.030542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:28.052524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:28.063943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1042/2094] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:28.106592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:28.108160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:28.117969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1053/2095] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:27 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:28.135986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:28.137979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:28.151611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575d5e2-8a17-4df5-a9d2-ffc18be94e7c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:28.160159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:28.160873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:28.161153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:28.161486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:28.170928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:28.172852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:28.183586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1043/2096] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:28.252219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:28.262598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9fe5-d586-4339-8c66-bdc30b0086b5 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:28.268329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:28.269126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:28.269401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:28.269669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:28.271473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:28.294128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:28.333957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:28.335437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338066da-95a4-43e9-8e03-caadfddd5306 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:28.346028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1054/2097] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:38.178087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:38.341078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=17 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:49.942505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:50.012257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c8de0-afd2-4653-b7ff-f045b7e77622 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:50.018177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:50.018793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:50.019070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:50.019339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] PATH_INFO: `/v3/users/42a7b55e562940a687ecee9495854985` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:50.020552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a7b55e562940a687ecee9495854985)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:50.022664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:50.072572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a7b55e562940a687ecee9495854985'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.082285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a7b55e562940a687ecee9495854985'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.082895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a7b55e562940a687ecee9495854985'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.096935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a7b55e562940a687ecee9495854985'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.104027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a7b55e562940a687ecee9495854985'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.110871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1044/2098] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:49 2022] DELETE /identity/v3/users/42a7b55e562940a687ecee9495854985 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:50.115665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:50.116159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:50.116159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:50.136734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:50.137961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:50.195738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1055/2099] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:50.205783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:50.207317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:50.272420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c39c4-21de-4863-b9da-9989dd3e884a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:50.276501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:50.277164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:50.277467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:50.277755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:50.287745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:50.289247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:50.300315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1045/2100] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:50.851873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:50.854949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:50.855072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:50.856143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:50.867311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fcd01-b9f4-4241-a578-95e2a8af5d0c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:50.870490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b910a-c0ea-4606-b836-f051dae3025e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:50.874114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:50.874963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:50.875106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:50.875324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] PATH_INFO: `/v3/projects/45434f8a513f4c53925da7a110c6d6f9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:50.875464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:50.875929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:50.876206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:50.876478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] PATH_INFO: `/v3/users/1bb74653b8784120859dc95daa62c66b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:50.876712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] RBAC: Authorizing `identity:delete_project(project_id=45434f8a513f4c53925da7a110c6d6f9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:50.877676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb74653b8784120859dc95daa62c66b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:50.880097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:50.880425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:50.899494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:50.936110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb74653b8784120859dc95daa62c66b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.944713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb74653b8784120859dc95daa62c66b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.945356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb74653b8784120859dc95daa62c66b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.951407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45434f8a513f4c53925da7a110c6d6f9', 'request_id': 'req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.957985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb74653b8784120859dc95daa62c66b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.958736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45434f8a513f4c53925da7a110c6d6f9', 'request_id': 'req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.967043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-69b86b6e-533f-4de2-8240-68b82b50a3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb74653b8784120859dc95daa62c66b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.971776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45434f8a513f4c53925da7a110c6d6f9', 'request_id': 'req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.973798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1046/2101] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:50 2022] DELETE /identity/v3/users/1bb74653b8784120859dc95daa62c66b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:50.979943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:50.980391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:50.980662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:50.990429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] The token cache is being invalidate because project 45434f8a513f4c53925da7a110c6d6f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:50.990714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.992231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:50.993869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1056/2102] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:50 2022] DELETE /identity/v3/projects/45434f8a513f4c53925da7a110c6d6f9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:51.000815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:51.004531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.009313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:51.011790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.074296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc38c9-fb0e-4c7c-8dcc-90c2706a661f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:51.083103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1047/2103] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:51.093588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:51.095178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.110300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:51.144044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:51.144916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.145072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.145773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] PATH_INFO: `/v3/users/77ef7235b6074df78b78bd786b1f5cdc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.146538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ef7235b6074df78b78bd786b1f5cdc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:51.148477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:51.158815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4be7-3b2d-49cc-928c-fcde243acf36 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:51.164309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:51.165271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.165523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.166018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.178001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:51.181463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:51.192250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1048/2104] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:38:51.217374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ef7235b6074df78b78bd786b1f5cdc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.230637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ef7235b6074df78b78bd786b1f5cdc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.231666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ef7235b6074df78b78bd786b1f5cdc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.243812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ef7235b6074df78b78bd786b1f5cdc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.244754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ef7235b6074df78b78bd786b1f5cdc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.267769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1057/2105] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:50 2022] DELETE /identity/v3/users/77ef7235b6074df78b78bd786b1f5cdc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:51.492361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:51.494025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.508309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eeffc9-f0d6-4240-bc32-214ac39a4e6e service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:51.545419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:51.564377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:51.567470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.574844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:51.575775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.576265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.576750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] PATH_INFO: `/v3/projects/1716a51c2dda40c498c5f501fe89cc44` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.578539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1716a51c2dda40c498c5f501fe89cc44)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:51.580644 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:51.632079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1716a51c2dda40c498c5f501fe89cc44', 'request_id': 'req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.638474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1716a51c2dda40c498c5f501fe89cc44', 'request_id': 'req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.641752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9cae-f5fc-47ef-b1f4-4e28c52855f9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:51.644731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1716a51c2dda40c498c5f501fe89cc44', 'request_id': 'req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.658376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] The token cache is being invalidate because project 1716a51c2dda40c498c5f501fe89cc44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:51.658664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.659267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bb8abdac-56d4-48b2-99eb-4d42c6bd399f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.660884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1049/2106] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:51 2022] DELETE /identity/v3/projects/1716a51c2dda40c498c5f501fe89cc44 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:51.696947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:51.704186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:51.705145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.705537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.705914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] PATH_INFO: `/v3/projects/f3cf7034871444938deed61c83713f02` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.707351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cf7034871444938deed61c83713f02)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:51.709994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:51.766778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cf7034871444938deed61c83713f02', 'request_id': 'req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.773949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cf7034871444938deed61c83713f02', 'request_id': 'req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.781320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cf7034871444938deed61c83713f02', 'request_id': 'req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.797152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] The token cache is being invalidate because project f3cf7034871444938deed61c83713f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:51.797641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.798336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a285a2ea-646b-4b5d-b7d2-a6a443bc2084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:51.799989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1058/2107] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:51 2022] DELETE /identity/v3/projects/f3cf7034871444938deed61c83713f02 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:51.806596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.806895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.807153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.827832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:51.829087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.899005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1050/2108] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:51.907514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.907799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.908073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.929321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:51.930676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:51.938787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1059/2109] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:51.943150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:51.952893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094d27c-535c-4927-818f-cad97057a9fd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:51.959354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:51.960530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.960955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.961298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:51.962205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:51.962804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:51.964731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:51.979330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1051/2110] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:51.983950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:51.993890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257156a-5c2d-461d-92c7-1172446fb125 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:51.998800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:51.999410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:51.999688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:51.999955 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.000647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.002610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.037199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1060/2111] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:51 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:38:52.038309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.059634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39724dc5-c70a-414b-b69b-bed913ea5d9e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.110577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:52.118449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.119111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.119399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.119683 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.120378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.122488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.147231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1052/2112] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:52 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:52.151599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.162907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35443f-a747-47d4-9590-fbaa3be3015a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.167612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.168220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.168504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.168748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.169470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.171045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.181349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1061/2113] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:52.185502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.195773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0974c19-e478-413a-be7f-f7d67e3e332f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.200857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dc591-7640-47dc-ab14-082e15e3cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.201142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc591-7640-47dc-ab14-082e15e3cea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.201418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc591-7640-47dc-ab14-082e15e3cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.201687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc591-7640-47dc-ab14-082e15e3cea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.202474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dc591-7640-47dc-ab14-082e15e3cea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.204072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dc591-7640-47dc-ab14-082e15e3cea6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.215278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1053/2114] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:52.219473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.231273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff3c43-c37b-4458-9bde-09e96da8d212 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.238148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.238828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.239106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.239377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] PATH_INFO: `/v3/projects/e1d0a5fe83e14a42a6cd85cf526b0f8f/users/e83d6d09ea8c4fcd89a7ee5ec84130cc/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.254946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d0a5fe83e14a42a6cd85cf526b0f8f, user_id=e83d6d09ea8c4fcd89a7ee5ec84130cc, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.257674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.277217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1062/2115] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:52 2022] PUT /identity/v3/projects/e1d0a5fe83e14a42a6cd85cf526b0f8f/users/e83d6d09ea8c4fcd89a7ee5ec84130cc/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:52.288949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.289312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.289651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.321189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] MFA Rules not processed for user `e83d6d09ea8c4fcd89a7ee5ec84130cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:52.322385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:52.398607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1054/2116] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:52.403177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.499129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ada90-11e0-4848-a0aa-be36243fed5a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.504710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.505458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.505824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.506044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.506733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.508625 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.547526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1063/2117] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:52 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:52.551230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.565066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043afc9-e3ab-498c-92c7-e67ed8b89a05 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.631423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.632233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.632579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.632962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.633751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.636046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.658655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1055/2118] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:52 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:52.662494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.672824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250c4b0-b719-4b65-a642-cbc9679a1fe0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.677431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.678063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.678335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.678608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.679280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.680905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.683171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:52.694454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1064/2119] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:52.699039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.710185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce52e8f-569a-48d0-945a-317586987c61 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.715684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.716073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.716258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.716529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] PATH_INFO: `/v3/projects/631635b8199f4d148b2a65293d3273c4/users/64a437a2696b4a879d7b30970e452e43/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.732578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=631635b8199f4d148b2a65293d3273c4, user_id=64a437a2696b4a879d7b30970e452e43, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.735324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.754042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1056/2120] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:52 2022] PUT /identity/v3/projects/631635b8199f4d148b2a65293d3273c4/users/64a437a2696b4a879d7b30970e452e43/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:52.758637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.769787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b5d4c-9a39-412c-a7eb-ead37ef11ad5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.827208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f6a4f-f440-4591-b9de-aa32d31fac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.827813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f6a4f-f440-4591-b9de-aa32d31fac49 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.828088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f6a4f-f440-4591-b9de-aa32d31fac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.828355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f6a4f-f440-4591-b9de-aa32d31fac49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.829076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f6a4f-f440-4591-b9de-aa32d31fac49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.830676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f6a4f-f440-4591-b9de-aa32d31fac49 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.844370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1065/2121] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:52.847156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:52.858746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6b293-6a36-494b-83af-a7377304d4ea admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:52.863645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:52.864328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.864607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.864921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] PATH_INFO: `/v3/projects/631635b8199f4d148b2a65293d3273c4/users/64a437a2696b4a879d7b30970e452e43/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.866979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] RBAC: Authorizing `identity:create_grant(project_id=631635b8199f4d148b2a65293d3273c4, user_id=64a437a2696b4a879d7b30970e452e43, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:52.869817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:52.882210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1057/2122] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:52 2022] PUT /identity/v3/projects/631635b8199f4d148b2a65293d3273c4/users/64a437a2696b4a879d7b30970e452e43/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:52.893359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:52.893790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:52.894131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:52.927953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] MFA Rules not processed for user `64a437a2696b4a879d7b30970e452e43`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:52.929879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:53.019193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1066/2123] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:53.030938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:53.032949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:53.054593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:53.093756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0af18-baf3-42d7-a6ea-0b38b56e346d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:53.097781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:53.098416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:53.098690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:53.098954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:53.108624 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:53.110282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:53.120154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1058/2124] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:53.206951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:53.216660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c385ec6-c0ec-4e2c-8125-9625d16cb7eb None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:53.220488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddc68e-8ce8-441f-9e29-3ec8c04b366a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:53.221141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc68e-8ce8-441f-9e29-3ec8c04b366a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:53.221414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc68e-8ce8-441f-9e29-3ec8c04b366a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:53.221683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc68e-8ce8-441f-9e29-3ec8c04b366a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:53.282387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddc68e-8ce8-441f-9e29-3ec8c04b366a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:53.283831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddc68e-8ce8-441f-9e29-3ec8c04b366a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:53.293575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1067/2125] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:56.062607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:56.125462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ec9c-9e0f-4fa2-959a-724e84854858 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:56.130727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:56.131400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:56.131673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:56.131943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] PATH_INFO: `/v3/users/e83d6d09ea8c4fcd89a7ee5ec84130cc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:56.133208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e83d6d09ea8c4fcd89a7ee5ec84130cc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:56.135123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:56.188798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e83d6d09ea8c4fcd89a7ee5ec84130cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.198190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e83d6d09ea8c4fcd89a7ee5ec84130cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.198887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e83d6d09ea8c4fcd89a7ee5ec84130cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.215659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e83d6d09ea8c4fcd89a7ee5ec84130cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.225581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e83d6d09ea8c4fcd89a7ee5ec84130cc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.232260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1059/2126] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:56 2022] DELETE /identity/v3/users/e83d6d09ea8c4fcd89a7ee5ec84130cc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:56.237268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:56.237773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:56.237873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:56.257509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:56.258690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:56.270809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:56.334058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1068/2127] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:56.344110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:56.345664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:56.391710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:56.409513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a47a-4679-4a13-808d-d4cd978ed147 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:56.413676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:56.414311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:56.414588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:56.414857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:56.424603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:56.426198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:56.436126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1060/2128] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:38:56.754937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:56.756414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:56.768564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396514c-5cd4-4bdd-a687-7dc1b4a0ce1c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:56.773782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:56.774553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:56.774895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:56.775238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] PATH_INFO: `/v3/projects/e1d0a5fe83e14a42a6cd85cf526b0f8f` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:56.776467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d0a5fe83e14a42a6cd85cf526b0f8f)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:56.778741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:56.828931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d0a5fe83e14a42a6cd85cf526b0f8f', 'request_id': 'req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.835086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d0a5fe83e14a42a6cd85cf526b0f8f', 'request_id': 'req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.841788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d0a5fe83e14a42a6cd85cf526b0f8f', 'request_id': 'req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.856813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] The token cache is being invalidate because project e1d0a5fe83e14a42a6cd85cf526b0f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:56.857105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.857734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.859270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1069/2129] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:56 2022] DELETE /identity/v3/projects/e1d0a5fe83e14a42a6cd85cf526b0f8f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:56.863779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:56.873946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e96bf6-01fa-465f-a45c-791185ef5324 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:56.935073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:56.935734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:56.935993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:56.936292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] PATH_INFO: `/v3/users/64a437a2696b4a879d7b30970e452e43` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:56.937559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a437a2696b4a879d7b30970e452e43)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:56.939580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:56.991609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a437a2696b4a879d7b30970e452e43'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.998798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a437a2696b4a879d7b30970e452e43'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:56.999399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a437a2696b4a879d7b30970e452e43'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.011484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a437a2696b4a879d7b30970e452e43'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.018471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bf5664a9-3c10-4dca-80de-7b89b1d5ed30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a437a2696b4a879d7b30970e452e43'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.024952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1061/2130] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:38:56 2022] DELETE /identity/v3/users/64a437a2696b4a879d7b30970e452e43 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:57.309950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.311793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:57.325280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a06b91-ad6e-4330-894d-b74e7c70a2b1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.383816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.384459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.384734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.385033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] PATH_INFO: `/v3/projects/631635b8199f4d148b2a65293d3273c4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.386226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631635b8199f4d148b2a65293d3273c4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.388150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.439547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:57.445579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631635b8199f4d148b2a65293d3273c4', 'request_id': 'req-d1e8d374-7b79-4a51-b842-be212fb79e10'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.452999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631635b8199f4d148b2a65293d3273c4', 'request_id': 'req-d1e8d374-7b79-4a51-b842-be212fb79e10'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.459550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631635b8199f4d148b2a65293d3273c4', 'request_id': 'req-d1e8d374-7b79-4a51-b842-be212fb79e10'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.477019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] The token cache is being invalidate because project 631635b8199f4d148b2a65293d3273c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:38:57.477310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.477958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-d1e8d374-7b79-4a51-b842-be212fb79e10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:38:57.479501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1070/2131] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:38:57 2022] DELETE /identity/v3/projects/631635b8199f4d148b2a65293d3273c4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:57.484496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.485007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.485105 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.504984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:57.506202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:57.566184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:57.576840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1062/2132] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:57.584894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.585192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.585465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.606029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:57.607224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:57.615790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1071/2133] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:57.620078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.630473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043cda4-3d1c-42dd-8a34-f604c5d0169e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.636321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.637329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.637726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.638044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.638828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:38:57.639269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.641056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.657307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1063/2134] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:38:57.662432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.672656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a107b1-4f77-4a88-8c7f-a9e22c8a0ea7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.678498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.679096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.679367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.679628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.680284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.682210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.715504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1072/2135] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:57 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:57.722146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.733473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01e9d0-261d-4e5d-b991-3f2030d431ce admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.807623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.808257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.808532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.808829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.809523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.811462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.835054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1064/2136] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:57 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:57.839113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.853569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4ce14-6cfe-4bd8-af6e-3689b717d858 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.858383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.859027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.859306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.859576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.860274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.861942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.872410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1073/2137] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:38:57.876591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.889657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561edb5a-f824-4cf7-ac7d-bd872802f33f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.895093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be418a-fae8-4566-a323-71b18ce57cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.895779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be418a-fae8-4566-a323-71b18ce57cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.896117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be418a-fae8-4566-a323-71b18ce57cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.897411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be418a-fae8-4566-a323-71b18ce57cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.897411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be418a-fae8-4566-a323-71b18ce57cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.899273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be418a-fae8-4566-a323-71b18ce57cf7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.910745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1065/2138] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:57.914911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:57.925038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f728f3f-4a8e-4846-8b8b-68b3d2436553 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:57.929711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:57.930367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.930654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.930923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] PATH_INFO: `/v3/projects/219cdfff67b14242a996a4bdfdc82387/users/3462179de95944b28b65dea8788a15f0/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:57.945217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219cdfff67b14242a996a4bdfdc82387, user_id=3462179de95944b28b65dea8788a15f0, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:57.947887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:57.962715 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1074/2139] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:57 2022] PUT /identity/v3/projects/219cdfff67b14242a996a4bdfdc82387/users/3462179de95944b28b65dea8788a15f0/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:57.975635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:57.976056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:57.976463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.007561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] MFA Rules not processed for user `3462179de95944b28b65dea8788a15f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:58.008810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:58.085528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1066/2140] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:58.090291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.156854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c920-9049-4e75-ae55-3f9e1cb3da93 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.162967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.163588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.163864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.164133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.164820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.166733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.196266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1075/2141] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:38:58 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:58.202069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.218619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53edefeb-c601-466c-9bee-14c289aa3d18 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.272101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:58.280193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.280844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.281125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.281391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.282044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.283974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.307235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1067/2142] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:38:58 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:38:58.311354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.322168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b23d1-5563-470a-9194-c9dcaf650e96 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.327105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.327705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.327980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.328244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.328975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.330571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.341242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1076/2143] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:58.345373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.355616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756354bd-ca32-4290-90d3-e7dce97a90a9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.360297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.361014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.361284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.361550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] PATH_INFO: `/v3/projects/a4f7d837531d4340af69684db4aa064c/users/1c3e5d43f4874c8686e0c61c40cab49c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.377135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f7d837531d4340af69684db4aa064c, user_id=1c3e5d43f4874c8686e0c61c40cab49c, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.380203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.395680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1068/2144] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:58 2022] PUT /identity/v3/projects/a4f7d837531d4340af69684db4aa064c/users/1c3e5d43f4874c8686e0c61c40cab49c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:38:58.400463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.412646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0901903-314d-4021-bb1c-8ea913b4a6b9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.453593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:38:58.473930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6106e7-e70f-4922-9d00-270cc303993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.474683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6106e7-e70f-4922-9d00-270cc303993e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.474958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6106e7-e70f-4922-9d00-270cc303993e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.475242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6106e7-e70f-4922-9d00-270cc303993e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.475914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6106e7-e70f-4922-9d00-270cc303993e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.477566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6106e7-e70f-4922-9d00-270cc303993e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.487322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1077/2145] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:38:58.491781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.504364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1098-992c-47aa-b334-9fc5f4a83047 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.511475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.512167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.512491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.512712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] PATH_INFO: `/v3/projects/a4f7d837531d4340af69684db4aa064c/users/1c3e5d43f4874c8686e0c61c40cab49c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.515532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f7d837531d4340af69684db4aa064c, user_id=1c3e5d43f4874c8686e0c61c40cab49c, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.519309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.533248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1069/2146] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:38:58 2022] PUT /identity/v3/projects/a4f7d837531d4340af69684db4aa064c/users/1c3e5d43f4874c8686e0c61c40cab49c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:38:58.544708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.545032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.545305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.575537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] MFA Rules not processed for user `1c3e5d43f4874c8686e0c61c40cab49c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:38:58.576751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:58.651232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1078/2147] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:38:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:38:58.662554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:38:58.664561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:38:58.726550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d018-9795-4f86-b23a-e470bcb183dd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:38:58.730967 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:38:58.731641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:38:58.731919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:38:58.732189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:38:58.790941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:38:58.792434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:38:58.802394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1070/2148] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:38:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:03.077672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:03.079256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:03.092529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a8d97-6d90-44d2-beed-6011d3396a81 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:03.096964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28df2e1-c427-423d-ab75-8039485eb098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:03.097592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28df2e1-c427-423d-ab75-8039485eb098 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:03.097865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28df2e1-c427-423d-ab75-8039485eb098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:03.098131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28df2e1-c427-423d-ab75-8039485eb098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:03.107838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28df2e1-c427-423d-ab75-8039485eb098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:03.109309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28df2e1-c427-423d-ab75-8039485eb098 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:03.118876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1079/2149] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:08.112987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:08.185968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a7344-c68f-4d1b-af07-1b7041cb5fba service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:08.190661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:08.191298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:08.191568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:08.191833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] PATH_INFO: `/v3/users/3462179de95944b28b65dea8788a15f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:08.193039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3462179de95944b28b65dea8788a15f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:08.194956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:08.197231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:08.250216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3462179de95944b28b65dea8788a15f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.258899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3462179de95944b28b65dea8788a15f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.259566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3462179de95944b28b65dea8788a15f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.273177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3462179de95944b28b65dea8788a15f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.280395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3462179de95944b28b65dea8788a15f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.286249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1071/2150] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:08 2022] DELETE /identity/v3/users/3462179de95944b28b65dea8788a15f0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:08.291012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:08.291466 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:08.291592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:08.311643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:08.312909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:08.373769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1080/2151] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:08.384057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:08.385548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:08.447193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49ec6-b40c-4736-9516-b4590ec33fdd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:08.451492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:08.452123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:08.452399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:08.452667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:08.464511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:08.466198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:08.476209 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1072/2152] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:39:08.815350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:08.817297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:08.831590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc289dc-6a02-4ec0-b262-2b98bac90b4c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:08.844288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:08.845102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:08.845380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:08.845649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] PATH_INFO: `/v3/projects/219cdfff67b14242a996a4bdfdc82387` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:08.848050 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=219cdfff67b14242a996a4bdfdc82387)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:08.850068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:08.909073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:08.913338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219cdfff67b14242a996a4bdfdc82387', 'request_id': 'req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.920976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219cdfff67b14242a996a4bdfdc82387', 'request_id': 'req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.927391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219cdfff67b14242a996a4bdfdc82387', 'request_id': 'req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.939907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] The token cache is being invalidate because project 219cdfff67b14242a996a4bdfdc82387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:08.940201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.940834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:08.942353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1081/2153] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:08 2022] DELETE /identity/v3/projects/219cdfff67b14242a996a4bdfdc82387 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:08.946568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:08.962330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce3b29-f78d-4902-ab50-99edf44af5d4 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.032090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.032737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.033212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.033487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] PATH_INFO: `/v3/users/1c3e5d43f4874c8686e0c61c40cab49c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.034661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3e5d43f4874c8686e0c61c40cab49c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.036663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.066884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:09.090614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3e5d43f4874c8686e0c61c40cab49c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.098755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3e5d43f4874c8686e0c61c40cab49c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.100565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3e5d43f4874c8686e0c61c40cab49c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.112789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3e5d43f4874c8686e0c61c40cab49c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.120571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96bfa10a-68bc-4145-98af-b8c247b1cbe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3e5d43f4874c8686e0c61c40cab49c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.126620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1073/2154] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:08 2022] DELETE /identity/v3/users/1c3e5d43f4874c8686e0c61c40cab49c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:09.406444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.408025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:09.421607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110fdfb-be68-4feb-aec3-2139cfb0dc3c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.485171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.486083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.486416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.486687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] PATH_INFO: `/v3/projects/a4f7d837531d4340af69684db4aa064c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.487923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f7d837531d4340af69684db4aa064c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.489952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.538092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f7d837531d4340af69684db4aa064c', 'request_id': 'req-116aec14-57a4-435c-a2ae-8b208148df0b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.544832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f7d837531d4340af69684db4aa064c', 'request_id': 'req-116aec14-57a4-435c-a2ae-8b208148df0b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.551006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f7d837531d4340af69684db4aa064c', 'request_id': 'req-116aec14-57a4-435c-a2ae-8b208148df0b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.563567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] The token cache is being invalidate because project a4f7d837531d4340af69684db4aa064c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:09.563851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.564489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-116aec14-57a4-435c-a2ae-8b208148df0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:09.566032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1082/2155] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:09 2022] DELETE /identity/v3/projects/a4f7d837531d4340af69684db4aa064c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:09.572189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.572469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.572736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.594793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:09.596029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:09.666653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1074/2156] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:09.675564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.675844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.676117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.697481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:09.698804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:09.708397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1083/2157] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:09.713043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.723902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0f807-1b07-4ebc-93ad-c5c8e8ee6b62 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.728999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.729637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.729910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.730177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.731640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:39:09.731969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.733636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.746660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1075/2158] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:39:09.751305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.761521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b612e-b218-41bc-9c66-d1640e2c6ac5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.766771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.767376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.767661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.769093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.769805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.771719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.800638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1084/2159] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:39:09 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:09.804982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.815589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e55cc-ba32-4c3c-9e61-c9677369c724 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.885610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.886225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.886498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.886768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.887471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.889513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.911794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1076/2160] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:39:09 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:09.916138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.927055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77a31-c3dc-435c-b47c-a5f312182486 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.932067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.932669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.932990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.933289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.933991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.935699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.945901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1085/2161] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:09.950035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.961582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec47a-e14b-40b2-acba-ea3c0e90e41f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:09.967177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50fd78-9ea5-47c2-84ee-2e8829431f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:09.967794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50fd78-9ea5-47c2-84ee-2e8829431f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:09.968076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50fd78-9ea5-47c2-84ee-2e8829431f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:09.968330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50fd78-9ea5-47c2-84ee-2e8829431f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:09.969051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50fd78-9ea5-47c2-84ee-2e8829431f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:09.970668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50fd78-9ea5-47c2-84ee-2e8829431f8f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:09.981994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1077/2162] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:09.986572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:09.997817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456a982-c386-40f1-bba9-7db560d1a5eb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:10.002774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:10.003446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.004087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.004367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] PATH_INFO: `/v3/projects/7c182af134ce4038acc53fb8dc94f545/users/8eab43a89bda4a33aa6b5720ba481357/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.019528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c182af134ce4038acc53fb8dc94f545, user_id=8eab43a89bda4a33aa6b5720ba481357, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:10.022239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:10.032559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:10.037862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1086/2163] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:39:09 2022] PUT /identity/v3/projects/7c182af134ce4038acc53fb8dc94f545/users/8eab43a89bda4a33aa6b5720ba481357/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:10.053632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f872f-2321-456a-b088-f23f79151f87 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.053922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f872f-2321-456a-b088-f23f79151f87 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.054185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f872f-2321-456a-b088-f23f79151f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.075708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:10.086885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-739f872f-2321-456a-b088-f23f79151f87 None None] MFA Rules not processed for user `8eab43a89bda4a33aa6b5720ba481357`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:10.088518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-739f872f-2321-456a-b088-f23f79151f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:10.155854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1078/2164] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:10.169118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:10.171815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:10.234741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4d4a-8006-4a36-86fe-ffab9de04a08 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:10.238722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6741d-a4f8-495e-b9d8-c84f82863877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:10.239330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6741d-a4f8-495e-b9d8-c84f82863877 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.239608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6741d-a4f8-495e-b9d8-c84f82863877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.239879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6741d-a4f8-495e-b9d8-c84f82863877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.249516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6741d-a4f8-495e-b9d8-c84f82863877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:10.250974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6741d-a4f8-495e-b9d8-c84f82863877 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:10.262650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1087/2165] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:10.284616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f872f-2321-456a-b088-f23f79151f87 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:10.356652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f872f-2321-456a-b088-f23f79151f87 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:10.361620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:10.362274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.362552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.362951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] PATH_INFO: `/v3/users/8eab43a89bda4a33aa6b5720ba481357` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.364324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eab43a89bda4a33aa6b5720ba481357)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:10.366338 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:10.416987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eab43a89bda4a33aa6b5720ba481357'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:10.424961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eab43a89bda4a33aa6b5720ba481357'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:10.425651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eab43a89bda4a33aa6b5720ba481357'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:10.438572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eab43a89bda4a33aa6b5720ba481357'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:10.445555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eab43a89bda4a33aa6b5720ba481357'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:10.452085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1079/2166] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:10 2022] DELETE /identity/v3/users/8eab43a89bda4a33aa6b5720ba481357 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:10.456354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.456859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.456960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.476358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:10.477575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:10.539616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1088/2167] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:10.549657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:10.551172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:10.572422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:10.610123 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15279aec-6cb5-4138-9c68-b80d5d325ecb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:10.614267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6cadc-087e-474b-92b0-26f2e08bb104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:10.614881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6cadc-087e-474b-92b0-26f2e08bb104 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.615154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6cadc-087e-474b-92b0-26f2e08bb104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.615426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6cadc-087e-474b-92b0-26f2e08bb104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.624798 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6cadc-087e-474b-92b0-26f2e08bb104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:10.626242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6cadc-087e-474b-92b0-26f2e08bb104 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:10.636226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1080/2168] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:10.927531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:10.929230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:10.943432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bd4a-c878-4a76-8dd0-d454f9a7f2a6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:10.950918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:10.950918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:10.950918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:10.950918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] PATH_INFO: `/v3/projects/7c182af134ce4038acc53fb8dc94f545` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:10.952058 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c182af134ce4038acc53fb8dc94f545)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:10.954040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.011525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c182af134ce4038acc53fb8dc94f545', 'request_id': 'req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:11.019854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c182af134ce4038acc53fb8dc94f545', 'request_id': 'req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:11.025648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c182af134ce4038acc53fb8dc94f545', 'request_id': 'req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:11.045280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] The token cache is being invalidate because project 7c182af134ce4038acc53fb8dc94f545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:11.045956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:11.046713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-af0fa912-c0e4-409e-a5ed-ee37c99e9ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:11.049415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1089/2169] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:10 2022] DELETE /identity/v3/projects/7c182af134ce4038acc53fb8dc94f545 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:11.054972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.055452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.055788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.083335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:11.084362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:11.152721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1081/2170] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:39:11.160442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.160723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.161027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.181196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:11.182399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:11.190607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1090/2171] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:39:11.194920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.205055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97f8cf-e855-4d6d-96cf-541ca2de7146 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.210184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.211507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.211807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.212088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.212895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:39:11.213109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.214697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.229184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1082/2172] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:39:11.234199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.244484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0b0a0-302a-4f68-a646-7e9001efa0d3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.249924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.250540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.250809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.251075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.251735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.254009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.281512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:11.289266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1091/2173] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:11.293394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.305873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9262ca-2a49-436f-904a-7dcc48510cd3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.365191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.365920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.366203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.366596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.367188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.369428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.394376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1083/2174] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:11.398683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.409307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971f1ac-7850-4006-a6d1-01b68e2cea4f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.413999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.414623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.414918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.415168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.415867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.417488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.429679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1092/2175] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:39:11.433905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.444394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbd109-edf7-4171-a793-5eaae765c443 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.450267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb453f0b-37be-4e54-a568-5d3afe5234c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.450889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453f0b-37be-4e54-a568-5d3afe5234c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.451188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453f0b-37be-4e54-a568-5d3afe5234c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.451456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453f0b-37be-4e54-a568-5d3afe5234c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.452148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453f0b-37be-4e54-a568-5d3afe5234c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.453772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453f0b-37be-4e54-a568-5d3afe5234c2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.483626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1084/2176] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:11.492627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.503451 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be5f3-7ea4-44cd-a596-3ebb76da67d0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.508618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.509422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.509703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.509977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] PATH_INFO: `/v3/projects/e32dd5b63f384092935a75a2d978c1ed/users/d4d78168b4994418b6f2dc0cfe50dce5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.525409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32dd5b63f384092935a75a2d978c1ed, user_id=d4d78168b4994418b6f2dc0cfe50dce5, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.528088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.544054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1093/2177] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:39:11 2022] PUT /identity/v3/projects/e32dd5b63f384092935a75a2d978c1ed/users/d4d78168b4994418b6f2dc0cfe50dce5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:39:11.554087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.554372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.554647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.588118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] MFA Rules not processed for user `d4d78168b4994418b6f2dc0cfe50dce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:11.589990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:11.610766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:11.675903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1085/2178] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:11.680742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.768565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b45e5-ab3e-4bdf-9295-a4d06dada5b4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.781827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.782490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.782754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.783030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.785720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.788294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.820858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1094/2179] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:11.824144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.837024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fb3f8-b851-4ce7-a7d6-df8d9d329b12 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.894929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.895556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.895829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.896100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.896779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.898776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.921027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1086/2180] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:39:11 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:39:11.925108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.935637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87901b-2fee-40cc-b36a-c96008f67a55 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.940605 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.941276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.941581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.941854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.942546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.944138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.955260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1095/2181] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:11.959653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:11.969816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd55f14-4945-408e-a704-5e5283068e62 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:11.974828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66a7ac-1bc2-4437-9385-88970f09f84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:11.975446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a7ac-1bc2-4437-9385-88970f09f84c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:11.975728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a7ac-1bc2-4437-9385-88970f09f84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:11.976028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a7ac-1bc2-4437-9385-88970f09f84c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:11.976684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66a7ac-1bc2-4437-9385-88970f09f84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:11.978316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66a7ac-1bc2-4437-9385-88970f09f84c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:11.989075 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1087/2182] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:39:11.993506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.005676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bc33a-d949-4f9a-8be8-cef20e64dd89 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.010865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.011544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.011800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.012144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] PATH_INFO: `/v3/projects/8b1b42a6e8274de3ad74c9e461cce4e9/users/a841b2956448443389d6e0dbbe9b20a5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.027533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1b42a6e8274de3ad74c9e461cce4e9, user_id=a841b2956448443389d6e0dbbe9b20a5, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.029826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.045948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1096/2183] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:39:11 2022] PUT /identity/v3/projects/8b1b42a6e8274de3ad74c9e461cce4e9/users/a841b2956448443389d6e0dbbe9b20a5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:12.058387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.058387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.058697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.100269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] MFA Rules not processed for user `a841b2956448443389d6e0dbbe9b20a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:12.101594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:12.185783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1088/2184] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:12.186676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.260106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34adaf-ef1d-4a5b-b666-44cd36db0fb3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.265723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.266431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.266769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.267123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.267868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.269806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.294895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:12.302972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1097/2185] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:39:12 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:39:12.307132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.317875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1990236-dc12-4d19-9986-299cb8d6d2b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.339498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:12.379100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.379716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.379990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.380256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.380957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.382880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.407143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1089/2186] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:39:12 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:12.412622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.424492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2603e98-d2fd-4908-a3f3-16af15b7123e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.429643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.430281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.430566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.430850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.431559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.433187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.443967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1098/2187] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:12.448354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.459119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e241f-4b86-4090-8c79-b74abb06838a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.464345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.465032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.465303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.465542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] PATH_INFO: `/v3/projects/24fc1c21283f4885b65adc46065f7fb9/users/2df88fd210ec4eb8a9224efd9427b5e6/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.481130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fc1c21283f4885b65adc46065f7fb9, user_id=2df88fd210ec4eb8a9224efd9427b5e6, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.483842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.499469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1090/2188] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:39:12 2022] PUT /identity/v3/projects/24fc1c21283f4885b65adc46065f7fb9/users/2df88fd210ec4eb8a9224efd9427b5e6/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:12.503949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.514413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674e139-674d-4ecd-a1ee-0a756a6957fa admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.572013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970810f-14c3-4cb5-bc30-a03017b9672d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.572644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970810f-14c3-4cb5-bc30-a03017b9672d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.572954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970810f-14c3-4cb5-bc30-a03017b9672d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.573228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970810f-14c3-4cb5-bc30-a03017b9672d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.573949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970810f-14c3-4cb5-bc30-a03017b9672d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.575671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970810f-14c3-4cb5-bc30-a03017b9672d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.585693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1099/2189] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:12.589954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.600250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4284139-a3f5-4fb3-8d8e-4aeecb274f68 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.605021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.605687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.605965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.606296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] PATH_INFO: `/v3/projects/24fc1c21283f4885b65adc46065f7fb9/users/2df88fd210ec4eb8a9224efd9427b5e6/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.608288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fc1c21283f4885b65adc46065f7fb9, user_id=2df88fd210ec4eb8a9224efd9427b5e6, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.610963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.621392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1091/2190] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:39:12 2022] PUT /identity/v3/projects/24fc1c21283f4885b65adc46065f7fb9/users/2df88fd210ec4eb8a9224efd9427b5e6/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:12.632030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.632456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.632901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.662757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] MFA Rules not processed for user `2df88fd210ec4eb8a9224efd9427b5e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:12.664001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:12.735422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1100/2191] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:39:12.747629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:12.749751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:12.812486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03f623-78bb-41c4-8bc0-db0aa625082a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:12.818037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:12.818673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:12.818952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:12.819221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:12.882505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:12.883945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:12.892970 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1092/2192] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:14.422421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:14.423990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:14.437164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e265-a14e-465c-b1ae-e5cb0ca83156 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:14.442269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:14.442977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:14.443252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:14.443519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:14.505669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:14.507108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:14.516582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1101/2193] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:15.077725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:15.086639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aef2b1-708f-478e-bc17-e3c6bae68423 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:15.090975 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af72a2-a7f4-4a6a-a43a-0014c9559732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:15.091578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af72a2-a7f4-4a6a-a43a-0014c9559732 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:15.091847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af72a2-a7f4-4a6a-a43a-0014c9559732 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:15.092112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af72a2-a7f4-4a6a-a43a-0014c9559732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:15.100938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af72a2-a7f4-4a6a-a43a-0014c9559732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:15.102399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af72a2-a7f4-4a6a-a43a-0014c9559732 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:15.112627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1093/2194] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:16.642844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:16.644282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:16.657129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:16.721226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121224c-18b1-4cde-98b0-3a5883c506a8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:16.726982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:16.727690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:16.727904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:16.728217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] PATH_INFO: `/v3/users/4814673c66784324987282c700e9bde2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:16.729876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4814673c66784324987282c700e9bde2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:16.732004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:16.806547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4814673c66784324987282c700e9bde2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:16.816981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4814673c66784324987282c700e9bde2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:16.818014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4814673c66784324987282c700e9bde2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:16.833599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4814673c66784324987282c700e9bde2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:16.839783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4814673c66784324987282c700e9bde2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:16.879106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1102/2195] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:16 2022] DELETE /identity/v3/users/4814673c66784324987282c700e9bde2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:16.883789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:16.884286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:16.884377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:16.905005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:16.906201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:16.925412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:16.998596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1094/2196] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:17.014203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:17.016365 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:17.103530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004f1ba-a46d-4bae-a409-2f1c10b309bb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:17.108292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13bd921-9f39-4266-b4f2-f978e481a50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:17.108978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bd921-9f39-4266-b4f2-f978e481a50f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:17.109585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bd921-9f39-4266-b4f2-f978e481a50f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:17.109873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bd921-9f39-4266-b4f2-f978e481a50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:17.129896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13bd921-9f39-4266-b4f2-f978e481a50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:17.131354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13bd921-9f39-4266-b4f2-f978e481a50f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:17.141497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1103/2197] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:39:17.426544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:17.428740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:17.448125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbf69c-d1ec-494f-982b-3bffee30750d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:17.461304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:17.462366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:17.462940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:17.463247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] PATH_INFO: `/v3/projects/2ccef57782ae4a38ab22206aa9289b23` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:17.464635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccef57782ae4a38ab22206aa9289b23)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:17.466800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:17.523309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccef57782ae4a38ab22206aa9289b23', 'request_id': 'req-d984f976-0532-455c-a603-0288b6defc37'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:17.529551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccef57782ae4a38ab22206aa9289b23', 'request_id': 'req-d984f976-0532-455c-a603-0288b6defc37'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:17.535555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccef57782ae4a38ab22206aa9289b23', 'request_id': 'req-d984f976-0532-455c-a603-0288b6defc37'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:17.550479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] The token cache is being invalidate because project 2ccef57782ae4a38ab22206aa9289b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:17.550764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:17.551527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:17.553130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1095/2198] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:17 2022] DELETE /identity/v3/projects/2ccef57782ae4a38ab22206aa9289b23 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:18.254612 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9734df-2ed3-4b81-98cc-bb85f4add899 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:18.255156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9734df-2ed3-4b81-98cc-bb85f4add899 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:18.255156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9734df-2ed3-4b81-98cc-bb85f4add899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:18.275544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3e9734df-2ed3-4b81-98cc-bb85f4add899 None None] MFA Rules not processed for user `88472fa130654657af599106c6aac279`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:18.277644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3e9734df-2ed3-4b81-98cc-bb85f4add899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:18.304732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:18.486535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1104/2199] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:19.081764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:19.083620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:19.155129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f976-0532-455c-a603-0288b6defc37 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:19.160443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:19.161131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:19.161402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:19.161668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] PATH_INFO: `/v3/users/d4d78168b4994418b6f2dc0cfe50dce5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:19.162871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d78168b4994418b6f2dc0cfe50dce5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:19.165167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:19.232919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:19.235981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d78168b4994418b6f2dc0cfe50dce5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:19.251944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d78168b4994418b6f2dc0cfe50dce5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:19.252686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d78168b4994418b6f2dc0cfe50dce5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:19.271303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d78168b4994418b6f2dc0cfe50dce5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:19.285826 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d78168b4994418b6f2dc0cfe50dce5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:19.293424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1096/2200] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:19 2022] DELETE /identity/v3/users/d4d78168b4994418b6f2dc0cfe50dce5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:19.298077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:19.298938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:19.299358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:19.320086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:19.321624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:19.420289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1105/2201] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:19.426897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:19.428508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:19.497737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc36f63-6608-4d0d-86f3-ee29aeca04ce admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:19.503829 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:19.504558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:19.504741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:19.505064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:19.517587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:19.519046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:19.530133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1097/2202] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:19.937962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:19.949265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:19.956283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbbac4-93c7-4417-a029-4e25a03c1438 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:19.962642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:19.962902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:19.969356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:19.969834 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] PATH_INFO: `/v3/projects/e32dd5b63f384092935a75a2d978c1ed` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:19.971360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32dd5b63f384092935a75a2d978c1ed)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:19.974966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:20.041799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32dd5b63f384092935a75a2d978c1ed', 'request_id': 'req-33b75a25-96d4-485d-9ea7-afa851710df4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.050480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32dd5b63f384092935a75a2d978c1ed', 'request_id': 'req-33b75a25-96d4-485d-9ea7-afa851710df4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.056748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32dd5b63f384092935a75a2d978c1ed', 'request_id': 'req-33b75a25-96d4-485d-9ea7-afa851710df4'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.072192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] The token cache is being invalidate because project e32dd5b63f384092935a75a2d978c1ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:20.072560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.073198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.075120 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1106/2203] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:19 2022] DELETE /identity/v3/projects/e32dd5b63f384092935a75a2d978c1ed => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:20.079331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:20.091281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc12572-5c89-4ea3-9a57-ef3f1d5d0acd service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:20.160351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:20.161039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:20.161313 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:20.161579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] PATH_INFO: `/v3/users/a841b2956448443389d6e0dbbe9b20a5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:20.162828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a841b2956448443389d6e0dbbe9b20a5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:20.164786 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:20.215060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a841b2956448443389d6e0dbbe9b20a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.233029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a841b2956448443389d6e0dbbe9b20a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.233827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a841b2956448443389d6e0dbbe9b20a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.242973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:20.249617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a841b2956448443389d6e0dbbe9b20a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.259234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a841b2956448443389d6e0dbbe9b20a5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.266257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1098/2204] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:20 2022] DELETE /identity/v3/users/a841b2956448443389d6e0dbbe9b20a5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:20.618001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:20.626841 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:20.669941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b75a25-96d4-485d-9ea7-afa851710df4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:20.748361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:20.759743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:20.760616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:20.761232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:20.761687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] PATH_INFO: `/v3/projects/8b1b42a6e8274de3ad74c9e461cce4e9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:20.762921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1b42a6e8274de3ad74c9e461cce4e9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:20.765020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:20.812145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1b42a6e8274de3ad74c9e461cce4e9', 'request_id': 'req-62936346-31ce-4d40-8cae-4e0abeafac07'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.819205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1b42a6e8274de3ad74c9e461cce4e9', 'request_id': 'req-62936346-31ce-4d40-8cae-4e0abeafac07'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.825823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1b42a6e8274de3ad74c9e461cce4e9', 'request_id': 'req-62936346-31ce-4d40-8cae-4e0abeafac07'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.840357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] The token cache is being invalidate because project 8b1b42a6e8274de3ad74c9e461cce4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:20.840639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.841336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:20.842974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1107/2205] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:20 2022] DELETE /identity/v3/projects/8b1b42a6e8274de3ad74c9e461cce4e9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:20.847546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:20.849181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:20.862455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf9ae-8485-438d-baf5-57608a6d2c86 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:20.925771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:20.926441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:20.926724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:20.926989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] PATH_INFO: `/v3/users/2df88fd210ec4eb8a9224efd9427b5e6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:20.928193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df88fd210ec4eb8a9224efd9427b5e6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:20.930222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:20.989367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df88fd210ec4eb8a9224efd9427b5e6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.002833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df88fd210ec4eb8a9224efd9427b5e6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.004140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df88fd210ec4eb8a9224efd9427b5e6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.023382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df88fd210ec4eb8a9224efd9427b5e6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.031074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-978f8db2-eaa4-47a5-abea-5a8a84d0bf55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df88fd210ec4eb8a9224efd9427b5e6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.037785 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1099/2206] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:39:20 2022] DELETE /identity/v3/users/2df88fd210ec4eb8a9224efd9427b5e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:21.521662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:21.523488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:21.538707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62936346-31ce-4d40-8cae-4e0abeafac07 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:21.614208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:21.614870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:21.615151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:21.615423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] PATH_INFO: `/v3/projects/24fc1c21283f4885b65adc46065f7fb9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:21.616750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fc1c21283f4885b65adc46065f7fb9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:21.618717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:21.671818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fc1c21283f4885b65adc46065f7fb9', 'request_id': 'req-1476125d-2506-46a1-a0ea-644eda55e458'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.678680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fc1c21283f4885b65adc46065f7fb9', 'request_id': 'req-1476125d-2506-46a1-a0ea-644eda55e458'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.684712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fc1c21283f4885b65adc46065f7fb9', 'request_id': 'req-1476125d-2506-46a1-a0ea-644eda55e458'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.700559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] The token cache is being invalidate because project 24fc1c21283f4885b65adc46065f7fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:39:21.700873 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.701504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-1476125d-2506-46a1-a0ea-644eda55e458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:39:21.706229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1108/2207] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:39:21 2022] DELETE /identity/v3/projects/24fc1c21283f4885b65adc46065f7fb9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:39:21.708043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:21.708493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:21.708581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:21.728619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:21.729948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:21.796471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1100/2208] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:21.805354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:21.805610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:21.805875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:21.825423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:21.826676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:21.834602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1109/2209] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:21.839017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:21.844068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:21.850632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b121e-609c-404d-a217-5f733a3e7d10 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:21.856625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:21.857302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:21.857577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:21.857851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:21.858545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:39:21.858846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:21.860458 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:21.872068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1101/2210] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:39:21.878105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:21.892353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a4a1c-f4a7-4c2a-8e0e-b377acd9fb53 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:21.899928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:21.900896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:21.901329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:21.901743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:21.902786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:21.905353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:21.915147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:21.936724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1110/2211] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:39:21 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:21.941751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:21.955328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34134fbc-9a2f-469a-834c-342adb9d22af admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:22.013710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:22.014328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:22.014601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:22.014867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:22.015520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:22.017490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:22.040233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1102/2212] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:39:21 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:39:22.044649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:22.055364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d9c17-9475-440b-af77-0dc0ef9d5eca admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:22.060918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:22.061531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:22.061808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:22.062080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:22.062794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:22.064372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:22.082246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1111/2213] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:22.088721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:22.103570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbccc3-2570-41b7-b095-fc9a39a9bf6b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:22.109523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762eb9a8-7477-4da0-a7cb-ec7382c7fbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:22.110137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eb9a8-7477-4da0-a7cb-ec7382c7fbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:22.110412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eb9a8-7477-4da0-a7cb-ec7382c7fbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:22.110678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eb9a8-7477-4da0-a7cb-ec7382c7fbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:22.111371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762eb9a8-7477-4da0-a7cb-ec7382c7fbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:22.113017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762eb9a8-7477-4da0-a7cb-ec7382c7fbe7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:22.124208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1103/2214] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:39:22.128792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:22.139530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86215b4-582f-47b5-ae06-555bedb9db26 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:22.145418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:22.146076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:22.146348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:22.146611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] PATH_INFO: `/v3/projects/91867ca69f334c76a3671331612ea954/users/0e0a63102acb43158fb074f468ca2faf/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:22.162211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91867ca69f334c76a3671331612ea954, user_id=0e0a63102acb43158fb074f468ca2faf, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:22.164890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:22.179888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1112/2215] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:39:22 2022] PUT /identity/v3/projects/91867ca69f334c76a3671331612ea954/users/0e0a63102acb43158fb074f468ca2faf/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:39:22.186692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3b53f-98f0-494b-8f67-04960f310d5f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:22.187007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3b53f-98f0-494b-8f67-04960f310d5f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:22.187276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3b53f-98f0-494b-8f67-04960f310d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:22.208282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-def3b53f-98f0-494b-8f67-04960f310d5f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:22.209558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-def3b53f-98f0-494b-8f67-04960f310d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:22.275725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1104/2216] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:22.286735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:22.288529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:22.358417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d4a5e-261d-4723-a0ef-47ac24560eb3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:22.363918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:22.364577 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:22.364943 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:22.365243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:22.375928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:22.377405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:22.388844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1113/2217] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:39:23.176102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c792-4379-43a0-be02-e64184700df9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:23.176621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c792-4379-43a0-be02-e64184700df9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:23.176885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c792-4379-43a0-be02-e64184700df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:23.207292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8761c792-4379-43a0-be02-e64184700df9 None None] MFA Rules not processed for user `0e0a63102acb43158fb074f468ca2faf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:23.208589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8761c792-4379-43a0-be02-e64184700df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:23.274570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:23.281421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1105/2218] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:39:23.292189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:23.293674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:23.350645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81035f0-d500-4429-bba4-aaa920355f39 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:23.354275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458fc01-8952-414e-8eb6-90f16f9a096f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:23.354898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458fc01-8952-414e-8eb6-90f16f9a096f service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:23.355204 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458fc01-8952-414e-8eb6-90f16f9a096f service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:23.355459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458fc01-8952-414e-8eb6-90f16f9a096f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:23.365023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458fc01-8952-414e-8eb6-90f16f9a096f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:23.366484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458fc01-8952-414e-8eb6-90f16f9a096f service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:23.376859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1114/2219] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:24.861360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f4776-944f-47ba-bbf7-d38f848923bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:24.861909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f4776-944f-47ba-bbf7-d38f848923bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:24.862065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f4776-944f-47ba-bbf7-d38f848923bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:24.885712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-936f4776-944f-47ba-bbf7-d38f848923bf None None] MFA Rules not processed for user `0e0a63102acb43158fb074f468ca2faf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:24.887017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-936f4776-944f-47ba-bbf7-d38f848923bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:24.895503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1106/2220] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:26.520952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:26.521548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:26.521548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:26.545312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] MFA Rules not processed for user `0d7ed577305d48c795f280e49b438bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:26.546808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:26.612911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1115/2221] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:32.232618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93725a9-92f6-44c9-ace0-9f02592ddd28 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:32.233124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93725a9-92f6-44c9-ace0-9f02592ddd28 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:32.233599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93725a9-92f6-44c9-ace0-9f02592ddd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:32.256280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e93725a9-92f6-44c9-ace0-9f02592ddd28 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:32.257672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e93725a9-92f6-44c9-ace0-9f02592ddd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:32.322765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1107/2222] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:39:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:32.367670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:32.369129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:32.376193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:32.446512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411c2af-ddce-4257-8744-99577f8d8d9e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:32.451964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ecb90-e189-4aed-9a58-175c738ae392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:32.452592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecb90-e189-4aed-9a58-175c738ae392 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:32.453192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecb90-e189-4aed-9a58-175c738ae392 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:32.453192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecb90-e189-4aed-9a58-175c738ae392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:32.461739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ecb90-e189-4aed-9a58-175c738ae392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:32.463328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ecb90-e189-4aed-9a58-175c738ae392 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:32.472236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1116/2223] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:39:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:39:42.311711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=29 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:42.470798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=30 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:39:44.806068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b84f7-35a8-49f6-80e8-989396278426 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:44.806955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b84f7-35a8-49f6-80e8-989396278426 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:44.807288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b84f7-35a8-49f6-80e8-989396278426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:44.829092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bf3b84f7-35a8-49f6-80e8-989396278426 None None] MFA Rules not processed for user `0e0a63102acb43158fb074f468ca2faf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:44.830520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bf3b84f7-35a8-49f6-80e8-989396278426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:44.837883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1108/2224] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:39:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:39:49.666083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33ffd-93f1-4b00-b97e-b507f52bd056 None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:49.666607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33ffd-93f1-4b00-b97e-b507f52bd056 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:49.666691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33ffd-93f1-4b00-b97e-b507f52bd056 None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:49.668282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1117/2225] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:39:49.674959 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e44d-c69c-4687-8481-ec474d3a4958 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:49.675455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e44d-c69c-4687-8481-ec474d3a4958 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:49.675455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e44d-c69c-4687-8481-ec474d3a4958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:49.696270 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9077e44d-c69c-4687-8481-ec474d3a4958 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:49.698172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9077e44d-c69c-4687-8481-ec474d3a4958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:49.772576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1109/2226] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:39:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:49.784150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33ffd-93f1-4b00-b97e-b507f52bd056 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:49.785993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fde33ffd-93f1-4b00-b97e-b507f52bd056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:49.850066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33ffd-93f1-4b00-b97e-b507f52bd056 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:49.853722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939fb85-853d-44dc-b3f4-cfa1de964fab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:49.854411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939fb85-853d-44dc-b3f4-cfa1de964fab service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:49.854756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939fb85-853d-44dc-b3f4-cfa1de964fab service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:49.855092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939fb85-853d-44dc-b3f4-cfa1de964fab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:49.865849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939fb85-853d-44dc-b3f4-cfa1de964fab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:49.867297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939fb85-853d-44dc-b3f4-cfa1de964fab service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:49.878666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1118/2227] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:49.950549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89b69-155f-4682-b0be-4d04a656682a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:49.950741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89b69-155f-4682-b0be-4d04a656682a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:49.951035 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89b69-155f-4682-b0be-4d04a656682a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:49.952435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1110/2228] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:39:49.957483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5cac1-363b-4d9d-ae75-f9dffd525f66 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:49.957764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5cac1-363b-4d9d-ae75-f9dffd525f66 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:49.958037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5cac1-363b-4d9d-ae75-f9dffd525f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:49.977719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d3d5cac1-363b-4d9d-ae75-f9dffd525f66 None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:49.978934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d3d5cac1-363b-4d9d-ae75-f9dffd525f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:49.985959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1119/2229] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:39:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:49.998582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f89b69-155f-4682-b0be-4d04a656682a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:50.010322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f89b69-155f-4682-b0be-4d04a656682a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:50.014261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c57bc6-ed97-4d2b-a740-162b85ec10c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:50.014969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c57bc6-ed97-4d2b-a740-162b85ec10c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:50.015318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c57bc6-ed97-4d2b-a740-162b85ec10c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:50.015595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c57bc6-ed97-4d2b-a740-162b85ec10c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:50.022835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c57bc6-ed97-4d2b-a740-162b85ec10c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:50.024257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c57bc6-ed97-4d2b-a740-162b85ec10c4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:50.036968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1111/2230] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:50.153153 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2000d1-7f9b-4b81-975c-323b19f6462c None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:50.153571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2000d1-7f9b-4b81-975c-323b19f6462c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:50.153798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2000d1-7f9b-4b81-975c-323b19f6462c None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:50.155820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1120/2231] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:39:50.161652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3c34-112d-4ba3-ad2b-c1bdb5ce0210 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:50.161960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3c34-112d-4ba3-ad2b-c1bdb5ce0210 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:50.162232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3c34-112d-4ba3-ad2b-c1bdb5ce0210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:50.182885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-749e3c34-112d-4ba3-ad2b-c1bdb5ce0210 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:50.184242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-749e3c34-112d-4ba3-ad2b-c1bdb5ce0210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:50.195244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1112/2232] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:39:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:50.205767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2000d1-7f9b-4b81-975c-323b19f6462c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:50.224903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2000d1-7f9b-4b81-975c-323b19f6462c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:50.229766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:50.230253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:50.230546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:50.230812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:50.241932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:50.243392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:50.255348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1121/2233] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:39:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:39:54.170518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:54.170974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:54.171074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:54.190677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:39:54.191906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:39:54.200076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1113/2234] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:39:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:39:54.215443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:39:54.227356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632afe-5bc9-495d-a10a-2d239d120f7d service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:39:54.239161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea050a71-9c19-4cad-bba7-f14b2a62b00f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:39:54.239779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea050a71-9c19-4cad-bba7-f14b2a62b00f service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:39:54.240056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea050a71-9c19-4cad-bba7-f14b2a62b00f service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:39:54.240323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea050a71-9c19-4cad-bba7-f14b2a62b00f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:39:54.249794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea050a71-9c19-4cad-bba7-f14b2a62b00f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:39:54.251379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea050a71-9c19-4cad-bba7-f14b2a62b00f service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:39:54.260202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1122/2235] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:39:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:40:04.185513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=46 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:04.256661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=41 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:17.515198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:17.516675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:17.592325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c9c79-74c4-4cca-b547-c26b45bf57ae None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:17.597371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:17.597998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:17.598274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:17.598541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:17.600255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:17.611567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:17.613189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:17.622847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1114/2236] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:40:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:40:18.486227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1bbcf-b44e-44bc-950f-1e1ba7e4ad2b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:18.488643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1bbcf-b44e-44bc-950f-1e1ba7e4ad2b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:18.490961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1bbcf-b44e-44bc-950f-1e1ba7e4ad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:18.511650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c5d1bbcf-b44e-44bc-950f-1e1ba7e4ad2b None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:18.512884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c5d1bbcf-b44e-44bc-950f-1e1ba7e4ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:18.521226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1123/2237] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:40:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:18.534561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:18.546439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74256737-2144-4acc-a7e0-13de3529acd8 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:18.552308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:18.552979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:18.553259 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:18.553532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:18.562219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:18.563878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:18.572279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1115/2238] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:40:21.193433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be50366-4865-4b0d-8ef7-0078a90076d9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:21.194443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be50366-4865-4b0d-8ef7-0078a90076d9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:21.194946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be50366-4865-4b0d-8ef7-0078a90076d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:21.215844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9be50366-4865-4b0d-8ef7-0078a90076d9 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:21.217441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9be50366-4865-4b0d-8ef7-0078a90076d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:21.230033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1124/2239] 10.222.0.49 () {58 vars in 967 bytes} [Sun Jul 31 18:40:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:21.246974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:21.258569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687f47-2e1f-4571-8902-121e0e0d9360 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:21.265031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:21.265688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:21.265963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:21.266232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:21.275743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:21.277481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:21.287227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1116/2240] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:40:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:40:31.210878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:31.283306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=44 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:34.469369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:34.470048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:34.470048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:34.498241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:34.499960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:34.511391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1125/2241] 10.222.0.49 () {58 vars in 967 bytes} [Sun Jul 31 18:40:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:34.525544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:34.545281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7298a-5265-416b-a128-bf654b81a46f service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:34.554876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:34.556611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:34.557286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:34.557841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:34.569531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:34.572152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:34.584392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1117/2242] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:40:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:40:36.722726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:36.738887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74083a67-fd1d-4bc2-8458-e299335242ee None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:36.743668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:36.744294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:36.744562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:36.744870 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] PATH_INFO: `/v3/users/0e0a63102acb43158fb074f468ca2faf` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:36.746034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e0a63102acb43158fb074f468ca2faf)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:36.747935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:36.797238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e0a63102acb43158fb074f468ca2faf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:36.805303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e0a63102acb43158fb074f468ca2faf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:36.806341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e0a63102acb43158fb074f468ca2faf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:36.814327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e0a63102acb43158fb074f468ca2faf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:36.814987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f8e454b5-bd06-4ea7-a2c3-71c1eaf5e089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e0a63102acb43158fb074f468ca2faf'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:36.835411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1126/2243] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:40:36 2022] DELETE /identity/v3/users/0e0a63102acb43158fb074f468ca2faf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:40:37.042585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.069859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b66c90-db0d-4cc8-9270-85407ecbfd17 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.147687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.148418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.148755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.149102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] PATH_INFO: `/v3/projects/91867ca69f334c76a3671331612ea954` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.150312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91867ca69f334c76a3671331612ea954)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.152320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.201597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91867ca69f334c76a3671331612ea954', 'request_id': 'req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:37.208174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91867ca69f334c76a3671331612ea954', 'request_id': 'req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:37.214140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91867ca69f334c76a3671331612ea954', 'request_id': 'req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:37.234322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] The token cache is being invalidate because project 91867ca69f334c76a3671331612ea954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:40:37.234614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:37.235247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f5ab2b67-b842-413d-9b7a-5162ee4d8e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:37.236982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1118/2244] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:40:37 2022] DELETE /identity/v3/projects/91867ca69f334c76a3671331612ea954 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:40:37.242329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.242716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.243056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.263772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:37.265039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:37.325951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1127/2245] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:37.334985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.335294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.335584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.355794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:37.357078 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:37.365116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1119/2246] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:37.369629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.380910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324f4e-06b1-4d09-8250-e0e7cef19a74 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.386103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.386724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.386995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.387257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.387948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:40:37.388243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.389983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.402200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1128/2247] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:40:37.406715 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.416649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cae948-985e-4942-a60a-3c0e064c6928 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.421448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.422072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.422346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.422611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.423300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.425292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.460792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1120/2248] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:40:37 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:40:37.462759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.472698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754023b9-1a96-456c-a7de-70e9014cc54b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.521118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:37.531480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.532130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.532420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.532699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.533484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.535779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.564623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1129/2249] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:40:37 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:40:37.569073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.580029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed04869-aabf-409d-a96a-c944aae263d6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.585299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.585936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.586214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.586483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.587179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.588794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.599394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1121/2250] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:40:37.603469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.614077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b74a0-d816-4cff-a103-ebf70eef6028 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.619357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c38d1-9666-4f85-96a8-1e62547ce285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.619967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c38d1-9666-4f85-96a8-1e62547ce285 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.620239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c38d1-9666-4f85-96a8-1e62547ce285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.620505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c38d1-9666-4f85-96a8-1e62547ce285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.621248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c38d1-9666-4f85-96a8-1e62547ce285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.622840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c38d1-9666-4f85-96a8-1e62547ce285 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.634115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1130/2251] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:40:37.638310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.648260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469a626-f163-4097-bb25-ec40689062ed admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.653225 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.653877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.654148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.654415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] PATH_INFO: `/v3/projects/4ae97760ed4a47d88ac07776fc149e95/users/f5fda5ec366f426ab6965f3549dac703/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.668866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae97760ed4a47d88ac07776fc149e95, user_id=f5fda5ec366f426ab6965f3549dac703, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.671510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.686512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1122/2252] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:40:37 2022] PUT /identity/v3/projects/4ae97760ed4a47d88ac07776fc149e95/users/f5fda5ec366f426ab6965f3549dac703/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:40:37.696378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.696658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.696958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.726652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] MFA Rules not processed for user `f5fda5ec366f426ab6965f3549dac703`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:37.727836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:37.792941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1131/2253] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:40:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:37.805300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:37.807391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:37.823641 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:37.870754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a96f87-b477-4182-8e32-574cabd2bc45 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:37.874912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebfbed-caee-4821-ac32-1b3954b408fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:37.875528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebfbed-caee-4821-ac32-1b3954b408fa service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:37.875802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebfbed-caee-4821-ac32-1b3954b408fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:37.876072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebfbed-caee-4821-ac32-1b3954b408fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:37.885419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebfbed-caee-4821-ac32-1b3954b408fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:37.886849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebfbed-caee-4821-ac32-1b3954b408fa service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:37.895735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1123/2254] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:40:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:40:40.472409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:40.550561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18702a-e5b1-42e0-ba86-524bd1a9578e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:40.554959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:40.555603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:40.555875 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:40.556161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] PATH_INFO: `/v3/users/f5fda5ec366f426ab6965f3549dac703` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:40.557320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5fda5ec366f426ab6965f3549dac703)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:40.559225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:40.615582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5fda5ec366f426ab6965f3549dac703'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:40.627143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5fda5ec366f426ab6965f3549dac703'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:40.627858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5fda5ec366f426ab6965f3549dac703'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:40.637413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5fda5ec366f426ab6965f3549dac703'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:40.638093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5fda5ec366f426ab6965f3549dac703'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:40.640555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:40.660096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1132/2255] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:40:40 2022] DELETE /identity/v3/users/f5fda5ec366f426ab6965f3549dac703 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:40:40.664827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:40.665328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:40.665516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:40.684993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:40.686356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:40.756648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1124/2256] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:40.766606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:40.768184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:40.831862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9131e5-9a65-431c-8603-5e467736666d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:40.835649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09666db3-03ae-4b21-8904-1b52c5b12e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:40.836311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09666db3-03ae-4b21-8904-1b52c5b12e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:40.836588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09666db3-03ae-4b21-8904-1b52c5b12e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:40.836869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09666db3-03ae-4b21-8904-1b52c5b12e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:40.846701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09666db3-03ae-4b21-8904-1b52c5b12e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:40.848197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09666db3-03ae-4b21-8904-1b52c5b12e35 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:40.858971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1133/2257] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:40:41.166752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:40:41.168407 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:41.182126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f0722-b421-43c5-b2e0-10cced57675d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:40:41.187919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:40:41.188696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:41.189331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:41.189750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] PATH_INFO: `/v3/projects/4ae97760ed4a47d88ac07776fc149e95` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:41.191255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae97760ed4a47d88ac07776fc149e95)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:40:41.194625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:40:41.263097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae97760ed4a47d88ac07776fc149e95', 'request_id': 'req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:41.270725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae97760ed4a47d88ac07776fc149e95', 'request_id': 'req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:41.277435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae97760ed4a47d88ac07776fc149e95', 'request_id': 'req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:41.295298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] The token cache is being invalidate because project 4ae97760ed4a47d88ac07776fc149e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:40:41.295593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:41.296221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-874f1a1e-28dd-4d8e-9ec1-bc9e0534e556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:40:41.297952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1125/2258] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:40:41 2022] DELETE /identity/v3/projects/4ae97760ed4a47d88ac07776fc149e95 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:40:43.965380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb630-ee68-49da-ad37-32df5a42e0d9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:40:43.965922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb630-ee68-49da-ad37-32df5a42e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:40:43.965922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb630-ee68-49da-ad37-32df5a42e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:40:43.986158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ce1cb630-ee68-49da-ad37-32df5a42e0d9 None None] MFA Rules not processed for user `88472fa130654657af599106c6aac279`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:40:43.987566 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ce1cb630-ee68-49da-ad37-32df5a42e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:40:44.045772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1134/2259] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:40:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:40:51.293575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:40:54.038611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=58,keystone:DELETE=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:41:11.472921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb927de-0e7e-4c81-a948-99a4002b52a5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:41:11.473409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb927de-0e7e-4c81-a948-99a4002b52a5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:41:11.473501 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb927de-0e7e-4c81-a948-99a4002b52a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:41:11.493582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5eb927de-0e7e-4c81-a948-99a4002b52a5 None None] MFA Rules not processed for user `120c06d6dee6425ea5366b363d5a5f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:41:11.494684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5eb927de-0e7e-4c81-a948-99a4002b52a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:41:11.551815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1126/2260] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:41:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:41:14.210418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85463d19-743b-4641-9261-9d380e5b13c8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:41:14.210995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85463d19-743b-4641-9261-9d380e5b13c8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:41:14.210995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85463d19-743b-4641-9261-9d380e5b13c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:41:14.232222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-85463d19-743b-4641-9261-9d380e5b13c8 None None] MFA Rules not processed for user `0d7ed577305d48c795f280e49b438bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:41:14.233531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-85463d19-743b-4641-9261-9d380e5b13c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:41:14.288899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1135/2261] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:41:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:41:21.544360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:41:24.282037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=25 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:41:39.149701 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:41:39.150221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:41:39.150333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:41:39.170414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] MFA Rules not processed for user `4dc88d9b0bc743119dcb7dc2fa8fdcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:41:39.171483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:41:39.236207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1127/2262] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:41:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:41:49.229080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:41:50.638114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:41:50.638654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:41:50.638654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:41:50.658185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] MFA Rules not processed for user `88472fa130654657af599106c6aac279`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:41:50.659412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:41:50.667245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1136/2263] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:41:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:00.653416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:09.666084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:09.667570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:09.726023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1de96b-2a63-4e43-8df2-67514d4c5f1b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:09.730208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:09.730886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:09.731162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:09.731425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:09.733681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:09.826664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:09.828129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:09.843172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1128/2264] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:11.023795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:11.025618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:11.038427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7204b21-2644-45f4-846f-48204cc7525f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:11.042853 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:11.043476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:11.043746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:11.044012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:11.045920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:11.059819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:11.061290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:11.072521 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1137/2265] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:18.268850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:18.277930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92b291-0c8a-4709-a314-176a29eee92a service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:18.282518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:18.283200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:18.283468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:18.283732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:18.285794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:18.350033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:18.351556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:18.361280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1129/2266] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:21.067587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:21.802357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:21.803989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:21.817871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c057b-16e6-4b25-8da0-791493265ec2 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:21.822606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:21.823240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:21.823512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:21.823815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] PATH_INFO: `/v3/users/0d7ed577305d48c795f280e49b438bfc` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:21.825026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7ed577305d48c795f280e49b438bfc)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:21.826947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:21.873528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7ed577305d48c795f280e49b438bfc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:21.881097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7ed577305d48c795f280e49b438bfc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:21.881818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7ed577305d48c795f280e49b438bfc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:21.888686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7ed577305d48c795f280e49b438bfc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:21.889397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-982c588e-d655-42a1-9e18-502800115719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7ed577305d48c795f280e49b438bfc'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:21.912125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1138/2267] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:21 2022] DELETE /identity/v3/users/0d7ed577305d48c795f280e49b438bfc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:22.103257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.113605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34533d4-dc28-4090-b7ab-c18722d90cf8 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.152163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:22.180250 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.180857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.181357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.181938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] PATH_INFO: `/v3/projects/df42ccb208664e04ac753b0c64b4fab7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.183626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df42ccb208664e04ac753b0c64b4fab7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.185570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.248523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df42ccb208664e04ac753b0c64b4fab7', 'request_id': 'req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:22.257117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df42ccb208664e04ac753b0c64b4fab7', 'request_id': 'req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:22.263964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df42ccb208664e04ac753b0c64b4fab7', 'request_id': 'req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:22.285307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] The token cache is being invalidate because project df42ccb208664e04ac753b0c64b4fab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:22.285670 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:22.286693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f6109d4e-ab14-4b2a-9f84-3971f03eca5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:22.288890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1130/2268] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:22 2022] DELETE /identity/v3/projects/df42ccb208664e04ac753b0c64b4fab7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:22.294095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.294939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.295216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.315880 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:22.317157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:22.379558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1139/2269] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:22.388027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.388335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.388602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.409386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:22.410636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:22.420203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1131/2270] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:22.424737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.435150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fceb0c-5f5b-4a1c-91ba-609252794bc7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.439867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.440551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.440859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.441131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.441836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:42:22.442140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.443763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.456699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1140/2271] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:42:22.461351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.471122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21f5fa-6d42-4323-8002-c9723d7dba4c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.477490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.478113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.478443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.478845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.479579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.481562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.514624 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1132/2272] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:22 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:22.518835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.529591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70b5fa-e531-4ae6-ae3e-28c7f751a9b9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.589344 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32888988-1970-4e11-b983-254840c2899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.589959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32888988-1970-4e11-b983-254840c2899b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.590236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32888988-1970-4e11-b983-254840c2899b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.590509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32888988-1970-4e11-b983-254840c2899b admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.591182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32888988-1970-4e11-b983-254840c2899b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.593122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32888988-1970-4e11-b983-254840c2899b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.601247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:22.620499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1141/2273] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:22 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:22.624778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.636685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a370fae-6d5d-461f-a4c6-9bdf26e3396e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.641807 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.642433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.642709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.642977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.643674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.645327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.656038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1133/2274] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:22.660337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32888988-1970-4e11-b983-254840c2899b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.675956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32888988-1970-4e11-b983-254840c2899b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.682306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc12cee-ba37-4935-8004-86844a83995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.682937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc12cee-ba37-4935-8004-86844a83995a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.683196 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc12cee-ba37-4935-8004-86844a83995a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.683465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc12cee-ba37-4935-8004-86844a83995a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.684166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc12cee-ba37-4935-8004-86844a83995a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.685876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc12cee-ba37-4935-8004-86844a83995a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.697081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1142/2275] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:22.701514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.712646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbe68-dd2e-44d4-8cef-8aa7ee6389d1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.718411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.719073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.719353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.719593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] PATH_INFO: `/v3/projects/f50ecb6d564240c993a6cee4315f22cf/users/8a10eed8bb4d4ba190b06b214ea06e7b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.735100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50ecb6d564240c993a6cee4315f22cf, user_id=8a10eed8bb4d4ba190b06b214ea06e7b, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.737806 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.755229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1134/2276] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:22 2022] PUT /identity/v3/projects/f50ecb6d564240c993a6cee4315f22cf/users/8a10eed8bb4d4ba190b06b214ea06e7b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:22.766774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.767177 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.767653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.807727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] MFA Rules not processed for user `8a10eed8bb4d4ba190b06b214ea06e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:22.808950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:22.889668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1143/2277] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:22.894513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:22.961097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c864b-6469-4fba-afdc-f2d9487898b7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:22.966616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:22.967239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:22.967525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:22.967792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:22.968439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:22.971227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:22.977746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:23.010572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1135/2278] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:22 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:23.014913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.026413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3978ed-1002-41a4-bd2c-39cf9efac68d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.084323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.084991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.085273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.085546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.086229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.088235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.114678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1144/2279] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:23 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:23.115935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.133587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d79ef-6ba3-4be2-bfbe-e836fbd97e82 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.139384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.139996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.140263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.140527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.141257 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.142858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.154084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1136/2280] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:42:23.159979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.175409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8930-571f-4685-8195-e053687fdc5c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.181928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.182582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.182856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.183130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] PATH_INFO: `/v3/projects/714c17cd087844c58053f935712aeffb/users/a4cd58ab70694e27956f1d803061af72/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.197708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714c17cd087844c58053f935712aeffb, user_id=a4cd58ab70694e27956f1d803061af72, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.200366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.216571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1145/2281] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:23 2022] PUT /identity/v3/projects/714c17cd087844c58053f935712aeffb/users/a4cd58ab70694e27956f1d803061af72/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:23.220568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.231190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6ff17-2da2-4be2-87d0-6a880b8598cf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.289557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee01c5-d895-4047-9ea9-5efe6ffef78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.290444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee01c5-d895-4047-9ea9-5efe6ffef78a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.290716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee01c5-d895-4047-9ea9-5efe6ffef78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.290978 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee01c5-d895-4047-9ea9-5efe6ffef78a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.291664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee01c5-d895-4047-9ea9-5efe6ffef78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.293289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee01c5-d895-4047-9ea9-5efe6ffef78a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.303356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1137/2282] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:42:23.307699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.323236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20372b0b-33c7-4ca6-8ece-78bb0a3f0400 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.331044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.331964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.332306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.332646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] PATH_INFO: `/v3/projects/714c17cd087844c58053f935712aeffb/users/a4cd58ab70694e27956f1d803061af72/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.335784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714c17cd087844c58053f935712aeffb, user_id=a4cd58ab70694e27956f1d803061af72, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.338679 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.349405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1146/2283] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:23 2022] PUT /identity/v3/projects/714c17cd087844c58053f935712aeffb/users/a4cd58ab70694e27956f1d803061af72/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:23.364058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.364396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.364693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.403937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] MFA Rules not processed for user `a4cd58ab70694e27956f1d803061af72`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:23.405187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:23.473261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1138/2284] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:23.485101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.487179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:23.513827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:23.552864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b90475-f5e9-4c16-9b05-b0b48b8846b9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.557111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9e3ff-2c6c-4701-8edb-0ad99b622d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.559306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9e3ff-2c6c-4701-8edb-0ad99b622d31 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.559306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9e3ff-2c6c-4701-8edb-0ad99b622d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.559306 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9e3ff-2c6c-4701-8edb-0ad99b622d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.567763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9e3ff-2c6c-4701-8edb-0ad99b622d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.569214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9e3ff-2c6c-4701-8edb-0ad99b622d31 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.579059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1147/2285] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:23.645306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.711096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:23.715682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaabeb-0d77-4720-83cb-6bb536313f12 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.724839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.725500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.725775 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.726486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] PATH_INFO: `/v3/users/8a10eed8bb4d4ba190b06b214ea06e7b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.727811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a10eed8bb4d4ba190b06b214ea06e7b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.729790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:23.776048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a10eed8bb4d4ba190b06b214ea06e7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:23.783620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a10eed8bb4d4ba190b06b214ea06e7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:23.784284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a10eed8bb4d4ba190b06b214ea06e7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:23.796079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a10eed8bb4d4ba190b06b214ea06e7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:23.802814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a10eed8bb4d4ba190b06b214ea06e7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:23.808685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1139/2286] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:23 2022] DELETE /identity/v3/users/8a10eed8bb4d4ba190b06b214ea06e7b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:23.813385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.813664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.813934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.833357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:23.834557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:23.894930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1148/2287] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:23.905023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:23.906457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:23.974136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf9e99-9a9b-4793-86f5-d55fd652e2c5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:23.980091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:23.981019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:23.981345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:23.981706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:23.995974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:23.997452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:24.007500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1140/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:24.298626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:24.300369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:24.315045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90b6a8-568b-4e24-8e67-2351b9195a45 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:24.320530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:24.321345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:24.321588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:24.321969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] PATH_INFO: `/v3/projects/f50ecb6d564240c993a6cee4315f22cf` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:24.323535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50ecb6d564240c993a6cee4315f22cf)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:24.325465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:24.376254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50ecb6d564240c993a6cee4315f22cf', 'request_id': 'req-c56870bf-211b-4436-ba48-66ed3f77245a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.383301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50ecb6d564240c993a6cee4315f22cf', 'request_id': 'req-c56870bf-211b-4436-ba48-66ed3f77245a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.389580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50ecb6d564240c993a6cee4315f22cf', 'request_id': 'req-c56870bf-211b-4436-ba48-66ed3f77245a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.403758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] The token cache is being invalidate because project f50ecb6d564240c993a6cee4315f22cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:24.404133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.404787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.406426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1149/2289] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:24 2022] DELETE /identity/v3/projects/f50ecb6d564240c993a6cee4315f22cf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:24.410625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:24.420801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676b029-6c7c-4820-abef-a2167b4797ec service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:24.478512 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:24.479165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:24.479444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:24.479750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] PATH_INFO: `/v3/users/a4cd58ab70694e27956f1d803061af72` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:24.480960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4cd58ab70694e27956f1d803061af72)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:24.482896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:24.505129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:24.536320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4cd58ab70694e27956f1d803061af72'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.544143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4cd58ab70694e27956f1d803061af72'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.545005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4cd58ab70694e27956f1d803061af72'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.557502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4cd58ab70694e27956f1d803061af72'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.565531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4cd58ab70694e27956f1d803061af72'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:24.571944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1141/2290] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:24 2022] DELETE /identity/v3/users/a4cd58ab70694e27956f1d803061af72 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:24.692157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:24.695207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:24.747243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:24.749581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:24.782485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:24.791066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56870bf-211b-4436-ba48-66ed3f77245a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:24.841146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3a293-8a01-4cf8-968f-5729ff621338 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:24.843281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:24.844065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:24.844351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:24.844719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:24.846606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:24.847457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:24.848128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:24.848388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:24.848832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:24.851269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:24.915414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:24.916435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:24.918448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:24.920063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1b215-bdce-45cf-83ae-9a9a36d4c835 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:24.926618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1150/2291] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:24.929181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:24.930839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1142/2292] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:42:24.939897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:25.014194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03d60-97e3-493d-a568-1b355d57ec7d service glance] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.021215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.021884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.022162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.022434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] PATH_INFO: `/v3/projects/714c17cd087844c58053f935712aeffb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.023621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] RBAC: Authorizing `identity:delete_project(project_id=714c17cd087844c58053f935712aeffb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.025590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.078890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714c17cd087844c58053f935712aeffb', 'request_id': 'req-306dab67-b427-45b4-93fd-204c5dedc76a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:25.085145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714c17cd087844c58053f935712aeffb', 'request_id': 'req-306dab67-b427-45b4-93fd-204c5dedc76a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:25.089087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:25.091322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714c17cd087844c58053f935712aeffb', 'request_id': 'req-306dab67-b427-45b4-93fd-204c5dedc76a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:25.118030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] The token cache is being invalidate because project 714c17cd087844c58053f935712aeffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:25.118469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:25.119387 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-306dab67-b427-45b4-93fd-204c5dedc76a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:25.121097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1151/2293] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:24 2022] DELETE /identity/v3/projects/714c17cd087844c58053f935712aeffb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:25.126201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0488d5-af90-43e4-aa2f-24063df78a00 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.126541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0488d5-af90-43e4-aa2f-24063df78a00 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.126820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0488d5-af90-43e4-aa2f-24063df78a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.148156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-dd0488d5-af90-43e4-aa2f-24063df78a00 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:25.149409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dd0488d5-af90-43e4-aa2f-24063df78a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:25.180428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:25.216096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1143/2294] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:25.224487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.224781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.225068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.244936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:25.246293 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:25.250018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.250362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.250637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.254753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1152/2295] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:25.258980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.268201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cfd7c-df8a-4598-ae1c-4699405fe6ba None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.272414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:25.272727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.273420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.273628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:25.273805 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.273997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.274690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:42:25.274995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.276593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.288570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1153/2296] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:42:25.292907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.302483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5783341-569e-47e4-b3ee-0ff4b118ae2f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.308153 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.308779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.309086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.309369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.310042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.311961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.335898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1144/2297] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:25.343189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1154/2298] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:25.348673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.361792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac476eb2-a0b8-4fbe-a2cc-c5af820b8f14 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.387364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.389244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:25.422170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.422840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.423120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.423392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.424050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.426028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.449996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1145/2299] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:25.454745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.464459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dcd79-9a03-4ad1-9c95-062dea27f5ba admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.465412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a72f7-eae7-404c-a450-62446da48a10 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.469366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.469963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.470238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.470507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.470855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f759e5-e9f8-4433-8dcc-4f90a2432003 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.471194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.471463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f759e5-e9f8-4433-8dcc-4f90a2432003 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.471741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f759e5-e9f8-4433-8dcc-4f90a2432003 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.472007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f759e5-e9f8-4433-8dcc-4f90a2432003 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.472790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.482953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1146/2300] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:42:25.486975 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.497046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076961-b499-4fa7-9b23-ef1e0e31696b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.501809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.502393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.503028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.503296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.503955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.505564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.516519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1147/2301] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:42:25.520914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.529802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f759e5-e9f8-4433-8dcc-4f90a2432003 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.531442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f759e5-e9f8-4433-8dcc-4f90a2432003 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.532625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1adb88-f5ec-4c81-bfd0-11c2ba17d69f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.537495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.538177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.538452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.538747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] PATH_INFO: `/v3/projects/edc34c81edbd4ef3bb735f71aacb9ff3/users/df52d57668c0450ead1294a62a0765ca/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.540084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1155/2302] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:42:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:42:25.554680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc34c81edbd4ef3bb735f71aacb9ff3, user_id=df52d57668c0450ead1294a62a0765ca, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.557461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.572851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1148/2303] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:25 2022] PUT /identity/v3/projects/edc34c81edbd4ef3bb735f71aacb9ff3/users/df52d57668c0450ead1294a62a0765ca/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:25.584152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.584437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.584709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.614920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] MFA Rules not processed for user `df52d57668c0450ead1294a62a0765ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:25.616327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:25.653774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:25.694930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1156/2304] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:25.696918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.723617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:25.764353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9e9aa-988b-4ad5-b9cb-0b6a2271f524 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.769253 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.769942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.770289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.770622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.771362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.775895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.813467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1149/2305] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:25.816702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.826237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e11d3c-2e3f-43c8-90f9-0eb5b6385d0f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.889462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.890230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.890580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.890984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.891758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.893820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.922758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1157/2306] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:25 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:25.926832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.943608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2882-3e50-4f82-85c5-74d8734cef13 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.949122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.949754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.950092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.950296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:25.951195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:25.952936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:25.966095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1150/2307] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:42:25.970496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:25.982130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8e558-d689-486c-bd26-5adb6493f3b9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:25.987043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:25.987724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:25.988006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:25.988281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] PATH_INFO: `/v3/projects/fdb49b50f106443ca44f7b5b9793d87d/users/50e05cbb9c1348e5ae0259e60b6dad36/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:26.007540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb49b50f106443ca44f7b5b9793d87d, user_id=50e05cbb9c1348e5ae0259e60b6dad36, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:26.014232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:26.032842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1158/2308] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:25 2022] PUT /identity/v3/projects/fdb49b50f106443ca44f7b5b9793d87d/users/50e05cbb9c1348e5ae0259e60b6dad36/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:42:26.036876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:26.048628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee74f-e78a-42f0-846f-16f294ed205b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:26.119620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9ce87-1d4f-4bcd-8914-4e1163edbeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:26.120277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9ce87-1d4f-4bcd-8914-4e1163edbeaa admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:26.120553 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9ce87-1d4f-4bcd-8914-4e1163edbeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:26.123464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9ce87-1d4f-4bcd-8914-4e1163edbeaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:26.123464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9ce87-1d4f-4bcd-8914-4e1163edbeaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:26.123464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9ce87-1d4f-4bcd-8914-4e1163edbeaa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:26.144560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:26.150119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1151/2309] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:26.156236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515b52-bc69-46a0-a6be-be5894f1ed61 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:26.164788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:26.165567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:26.165942 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:26.166302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] PATH_INFO: `/v3/projects/fdb49b50f106443ca44f7b5b9793d87d/users/50e05cbb9c1348e5ae0259e60b6dad36/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:26.168376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb49b50f106443ca44f7b5b9793d87d, user_id=50e05cbb9c1348e5ae0259e60b6dad36, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:26.171169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:26.191367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1159/2310] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:26 2022] PUT /identity/v3/projects/fdb49b50f106443ca44f7b5b9793d87d/users/50e05cbb9c1348e5ae0259e60b6dad36/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:26.203113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b692d-f81e-46af-af14-87938112cc0a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:26.203393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b692d-f81e-46af-af14-87938112cc0a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:26.203665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b692d-f81e-46af-af14-87938112cc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:26.256074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-813b692d-f81e-46af-af14-87938112cc0a None None] MFA Rules not processed for user `50e05cbb9c1348e5ae0259e60b6dad36`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:26.257353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-813b692d-f81e-46af-af14-87938112cc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:26.336936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1152/2311] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:26.347367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:26.352895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:26.450112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7bf63-4868-425a-9991-a8dccbba25fb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:26.455485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88070ba-e304-40b8-8b57-7d3dc1dd14d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:26.456130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88070ba-e304-40b8-8b57-7d3dc1dd14d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:26.456892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88070ba-e304-40b8-8b57-7d3dc1dd14d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:26.457178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88070ba-e304-40b8-8b57-7d3dc1dd14d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:26.471109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88070ba-e304-40b8-8b57-7d3dc1dd14d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:26.472698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88070ba-e304-40b8-8b57-7d3dc1dd14d3 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:26.484872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1160/2312] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:27.145016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b692d-f81e-46af-af14-87938112cc0a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:27.176789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:27.238112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b692d-f81e-46af-af14-87938112cc0a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:27.245368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:27.247188 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:27.247685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:27.247960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] PATH_INFO: `/v3/users/df52d57668c0450ead1294a62a0765ca` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:27.249664 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df52d57668c0450ead1294a62a0765ca)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:27.251707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:27.312083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df52d57668c0450ead1294a62a0765ca'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.322147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df52d57668c0450ead1294a62a0765ca'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.323143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df52d57668c0450ead1294a62a0765ca'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.336934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df52d57668c0450ead1294a62a0765ca'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.344797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df52d57668c0450ead1294a62a0765ca'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.355335 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1153/2313] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:27 2022] DELETE /identity/v3/users/df52d57668c0450ead1294a62a0765ca => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:27.357372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:27.357897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:27.358089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:27.379495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:27.380780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:27.409403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:27.449652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1161/2314] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:27.460168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:27.461627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:27.544815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eb724-6f09-4aed-b611-de065341828c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:27.544815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:27.544815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:27.544815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:27.545266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:27.555844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:27.557417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:27.568230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1154/2315] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:27.877945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:27.879818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:27.895539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e8d8-7cd8-4ab1-b000-17f4b643d37a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:27.909017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:27.909673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:27.910038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:27.910192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] PATH_INFO: `/v3/projects/edc34c81edbd4ef3bb735f71aacb9ff3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:27.911559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc34c81edbd4ef3bb735f71aacb9ff3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:27.913302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:27.971658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc34c81edbd4ef3bb735f71aacb9ff3', 'request_id': 'req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.981820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc34c81edbd4ef3bb735f71aacb9ff3', 'request_id': 'req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:27.989017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc34c81edbd4ef3bb735f71aacb9ff3', 'request_id': 'req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.007598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] The token cache is being invalidate because project edc34c81edbd4ef3bb735f71aacb9ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:28.008299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.009212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.011338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1162/2316] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:27 2022] DELETE /identity/v3/projects/edc34c81edbd4ef3bb735f71aacb9ff3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:28.015638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:28.026564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339387c-9adb-4c37-873f-03234f97ad85 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:28.073584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:28.092336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:28.093026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:28.093354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:28.093631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] PATH_INFO: `/v3/users/50e05cbb9c1348e5ae0259e60b6dad36` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:28.094816 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50e05cbb9c1348e5ae0259e60b6dad36)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:28.096927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:28.157276 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50e05cbb9c1348e5ae0259e60b6dad36'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.166960 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50e05cbb9c1348e5ae0259e60b6dad36'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.167680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50e05cbb9c1348e5ae0259e60b6dad36'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.182834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50e05cbb9c1348e5ae0259e60b6dad36'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.191073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0e88397-1942-4208-a4d8-7471d0d2fb41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50e05cbb9c1348e5ae0259e60b6dad36'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.198021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1155/2317] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:28 2022] DELETE /identity/v3/users/50e05cbb9c1348e5ae0259e60b6dad36 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:28.543230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:28.544847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:28.563185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0daae3-33ad-4bfc-b5ff-8d69d431aeda admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:28.654604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:28.655410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:28.655688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:28.656256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] PATH_INFO: `/v3/projects/fdb49b50f106443ca44f7b5b9793d87d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:28.657453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb49b50f106443ca44f7b5b9793d87d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:28.659629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:28.738698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb49b50f106443ca44f7b5b9793d87d', 'request_id': 'req-f5422bc0-64c9-494b-b130-30e327db186b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.749623 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb49b50f106443ca44f7b5b9793d87d', 'request_id': 'req-f5422bc0-64c9-494b-b130-30e327db186b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.753310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:28.758020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb49b50f106443ca44f7b5b9793d87d', 'request_id': 'req-f5422bc0-64c9-494b-b130-30e327db186b'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.784958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] The token cache is being invalidate because project fdb49b50f106443ca44f7b5b9793d87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:28.785277 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.786049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f5422bc0-64c9-494b-b130-30e327db186b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:28.787866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1163/2318] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:28 2022] DELETE /identity/v3/projects/fdb49b50f106443ca44f7b5b9793d87d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:28.793269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:28.793754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:28.794528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:28.816459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:28.818057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:28.889030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1156/2319] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:28.897024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:28.897314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:28.897591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:28.920745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:28.922096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:28.931687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1164/2320] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:28.936081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:28.948252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9623ee6-c93d-47c4-aa1c-a3d13ac33298 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:28.954547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:28.955417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:28.955881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:28.956154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:28.957267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:42:28.957570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:28.959373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:28.971899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1157/2321] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:42:28.977433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:28.988005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f190be0-49f5-4225-90e9-0a8aa5a28116 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:28.992661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:28.993320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:28.993597 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:28.993867 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:28.994534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:28.996449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.033944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1165/2322] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:28 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:29.035128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.046411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aa629-8b9b-4610-9ad0-6469d26b9225 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.119856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:29.137204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.138130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.138409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.138677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.139339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.141322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.170360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1158/2323] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:29 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:29.171569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.182223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaa070-50b0-4ca0-aab5-02a7928a4ca2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.187876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.188224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.188910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.188910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.189544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.191028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.205670 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1166/2324] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:42:29.206896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.220904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd275-52f8-464e-b429-1fc7c83c43ac admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.226930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6e08d-ee38-46da-ac17-77c8d4df50c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.227703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e08d-ee38-46da-ac17-77c8d4df50c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.227985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e08d-ee38-46da-ac17-77c8d4df50c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.228260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e08d-ee38-46da-ac17-77c8d4df50c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.228999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e08d-ee38-46da-ac17-77c8d4df50c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.230620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e08d-ee38-46da-ac17-77c8d4df50c0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.245262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1159/2325] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:29.247903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.259504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc70712-a708-4a45-a88e-42ea354e01d8 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.266080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.266919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.267191 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.267461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] PATH_INFO: `/v3/projects/9e547f0b7a0349fab7cf7c64d1eaa853/users/dc6d640594a54fa382feffc6c53febb3/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.284728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e547f0b7a0349fab7cf7c64d1eaa853, user_id=dc6d640594a54fa382feffc6c53febb3, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.287483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.303024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1167/2326] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:29 2022] PUT /identity/v3/projects/9e547f0b7a0349fab7cf7c64d1eaa853/users/dc6d640594a54fa382feffc6c53febb3/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:29.314610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.314910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.315001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.349386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] MFA Rules not processed for user `dc6d640594a54fa382feffc6c53febb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:29.350124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:29.432651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1160/2327] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:29.433877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.518512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3c9d0-82d0-4cd6-942f-2c3ba7b55edd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.523314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.523981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.524264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.524546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.525257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.527172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.564036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1168/2328] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:29 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:29.565346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.579955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db7a43-7eb9-4d3e-ba66-40ebe77c2f17 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.657171 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.657812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.658094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.658370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.659048 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.661049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.688574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1161/2329] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:29 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:29.690118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.706288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364988d-7d26-4764-8aa5-3501e759a58f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.712428 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.713133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.713133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.713346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.714134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.715645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.727728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1169/2330] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:29.731790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.746841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8209-0d27-41c8-bb35-04284945744c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.751842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.753049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.753341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.753609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] PATH_INFO: `/v3/projects/4a2d8e28b3f34806889036f1d70ebac6/users/fabda0337e4046da8c1069b7f11a50c7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.772973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d8e28b3f34806889036f1d70ebac6, user_id=fabda0337e4046da8c1069b7f11a50c7, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.775621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.797214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1162/2331] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:29 2022] PUT /identity/v3/projects/4a2d8e28b3f34806889036f1d70ebac6/users/fabda0337e4046da8c1069b7f11a50c7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:42:29.799711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.814098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92ecce-c606-4e07-8f18-26a82269c875 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.908380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:29.919561 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a56ca6-7e00-4f56-b1bd-645137ff956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.921905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a56ca6-7e00-4f56-b1bd-645137ff956e admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.922259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a56ca6-7e00-4f56-b1bd-645137ff956e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.922596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a56ca6-7e00-4f56-b1bd-645137ff956e admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.923630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a56ca6-7e00-4f56-b1bd-645137ff956e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.925539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a56ca6-7e00-4f56-b1bd-645137ff956e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:29.945102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1170/2332] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:29.951059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:29.966148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a54778-e5c8-4596-9da0-0155fe0d156f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:29.971263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:29.971944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:29.972223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:29.972627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] PATH_INFO: `/v3/projects/4a2d8e28b3f34806889036f1d70ebac6/users/fabda0337e4046da8c1069b7f11a50c7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:29.975268 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d8e28b3f34806889036f1d70ebac6, user_id=fabda0337e4046da8c1069b7f11a50c7, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:29.978204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:30.004366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1163/2333] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:29 2022] PUT /identity/v3/projects/4a2d8e28b3f34806889036f1d70ebac6/users/fabda0337e4046da8c1069b7f11a50c7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:30.017508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:30.017903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:30.018156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:30.051952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] MFA Rules not processed for user `fabda0337e4046da8c1069b7f11a50c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:30.053263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:30.131958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1171/2334] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:30.143378 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:30.145129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:30.157337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:30.217066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395d43-21ac-4776-8f3b-3d7258484237 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:30.221457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c407c-d828-479c-bcdd-df31a70498d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:30.222410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c407c-d828-479c-bcdd-df31a70498d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:30.222833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c407c-d828-479c-bcdd-df31a70498d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:30.223451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c407c-d828-479c-bcdd-df31a70498d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:30.244047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c407c-d828-479c-bcdd-df31a70498d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:30.247182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c407c-d828-479c-bcdd-df31a70498d0 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:30.262949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1164/2335] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:30.697288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:30.769727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1145d-c457-44ef-8d68-356ba99be935 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:30.774728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:30.775412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:30.775694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:30.775987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] PATH_INFO: `/v3/users/dc6d640594a54fa382feffc6c53febb3` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:30.777189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6d640594a54fa382feffc6c53febb3)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:30.779121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:30.845891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6d640594a54fa382feffc6c53febb3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:30.854421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6d640594a54fa382feffc6c53febb3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:30.855092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6d640594a54fa382feffc6c53febb3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:30.863543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6d640594a54fa382feffc6c53febb3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:30.870653 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6d640594a54fa382feffc6c53febb3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:30.884459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:30.894755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1172/2336] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:30 2022] DELETE /identity/v3/users/dc6d640594a54fa382feffc6c53febb3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:30.900207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:30.900678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:30.901147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:30.921841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:30.923129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:31.021930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1165/2337] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:31.032806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:31.035103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:31.121042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a796a4-43de-4c8c-a15e-619e60b31c01 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:31.125091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:31.125731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:31.126008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:31.126285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:31.138254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:31.139624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:31.152798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1173/2338] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:31.471818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:31.473421 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:31.487068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4b8fe-4d85-4e58-8358-f864ebc34fce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:31.496364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:31.497828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:31.498486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:31.498595 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] PATH_INFO: `/v3/projects/9e547f0b7a0349fab7cf7c64d1eaa853` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:31.500647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e547f0b7a0349fab7cf7c64d1eaa853)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:31.503359 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:31.561377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e547f0b7a0349fab7cf7c64d1eaa853', 'request_id': 'req-77559403-1e64-44db-9fb5-78ce62cabbda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.569452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e547f0b7a0349fab7cf7c64d1eaa853', 'request_id': 'req-77559403-1e64-44db-9fb5-78ce62cabbda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.578918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e547f0b7a0349fab7cf7c64d1eaa853', 'request_id': 'req-77559403-1e64-44db-9fb5-78ce62cabbda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.596741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] The token cache is being invalidate because project 9e547f0b7a0349fab7cf7c64d1eaa853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:31.597046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.597805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.599397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1166/2339] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:31 2022] DELETE /identity/v3/projects/9e547f0b7a0349fab7cf7c64d1eaa853 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:31.603614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:31.616116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404d261-f631-42f9-95bd-c62ba767ff9b service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:31.682752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:31.684092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:31.684373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:31.684661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] PATH_INFO: `/v3/users/fabda0337e4046da8c1069b7f11a50c7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:31.685874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fabda0337e4046da8c1069b7f11a50c7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:31.687802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:31.741129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fabda0337e4046da8c1069b7f11a50c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.751948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fabda0337e4046da8c1069b7f11a50c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.753412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fabda0337e4046da8c1069b7f11a50c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.762232 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fabda0337e4046da8c1069b7f11a50c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.762971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8b6a1413-daba-40d3-b2ca-620ca45c9ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fabda0337e4046da8c1069b7f11a50c7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:31.784499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1174/2340] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:31 2022] DELETE /identity/v3/users/fabda0337e4046da8c1069b7f11a50c7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:32.074513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.076058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:32.091377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77559403-1e64-44db-9fb5-78ce62cabbda admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.137813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:32.166506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.167545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.167823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.168093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] PATH_INFO: `/v3/projects/4a2d8e28b3f34806889036f1d70ebac6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.169328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2d8e28b3f34806889036f1d70ebac6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.171386 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.244946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2d8e28b3f34806889036f1d70ebac6', 'request_id': 'req-495e6b9d-573b-417b-abcb-780c72588dd0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:32.254680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2d8e28b3f34806889036f1d70ebac6', 'request_id': 'req-495e6b9d-573b-417b-abcb-780c72588dd0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:32.265491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2d8e28b3f34806889036f1d70ebac6', 'request_id': 'req-495e6b9d-573b-417b-abcb-780c72588dd0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:32.284041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] The token cache is being invalidate because project 4a2d8e28b3f34806889036f1d70ebac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:32.284406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:32.285008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-495e6b9d-573b-417b-abcb-780c72588dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:32.287072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1167/2341] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:32 2022] DELETE /identity/v3/projects/4a2d8e28b3f34806889036f1d70ebac6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:32.291898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.292254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.292591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.324338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:32.326245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:32.341924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:32.425083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1175/2342] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:32.432958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.433309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.433669 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.461272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:32.462589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:32.471179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1168/2343] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:32.475271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.485826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17140fe0-cd86-4f31-affd-23ce782ee70f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.491393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.492015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.492323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.492706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.493448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:42:32.493901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.495514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.510698 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1176/2344] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:42:32.515463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.525044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9192c4-c24a-4fb1-974e-8ec38aede0d6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.529778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.530380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.530660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.530932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.531605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.533585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.566729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1169/2345] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:32 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:32.570872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.581594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e05d3-4484-430f-bcc7-66a61ce00fa9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.641092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.642161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.642590 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.643019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.643703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.646007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.670395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1177/2346] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:32 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:42:32.674372 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.685472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa55fc-36d3-4810-9db8-57c843fccdfb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.690616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.691445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.691575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.691797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.692520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.694449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.706131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1170/2347] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:32.710024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.721039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936695f-8c91-4fa8-80a7-1839076521bd admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.725706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502fdc8-fcc8-479c-8437-dc2e8c4cf9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.726329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502fdc8-fcc8-479c-8437-dc2e8c4cf9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.726609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502fdc8-fcc8-479c-8437-dc2e8c4cf9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.726946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502fdc8-fcc8-479c-8437-dc2e8c4cf9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.727647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502fdc8-fcc8-479c-8437-dc2e8c4cf9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.729408 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502fdc8-fcc8-479c-8437-dc2e8c4cf9a2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.744128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1178/2348] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:32.745242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.756010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49aa25-b922-4831-be4c-94777fbd8904 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.761274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.761962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.762247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.762497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] PATH_INFO: `/v3/projects/e46acd2afd554f28b528ef26cf02a484/users/2cf9c70427b54be39b99b1887a244c54/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.778673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46acd2afd554f28b528ef26cf02a484, user_id=2cf9c70427b54be39b99b1887a244c54, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:32.781365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:32.803537 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1171/2349] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:32 2022] PUT /identity/v3/projects/e46acd2afd554f28b528ef26cf02a484/users/2cf9c70427b54be39b99b1887a244c54/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:42:32.808672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.809069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.809341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.841329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] MFA Rules not processed for user `2cf9c70427b54be39b99b1887a244c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:32.842218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:32.923073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1179/2350] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:32.923988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:32.993025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e978d0-99ab-49f4-b3de-89fd48c84a5c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:32.997901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:32.998489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:32.998751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:32.999023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:32.999815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.001795 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.007906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:33.038161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1172/2351] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:32 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:33.042246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:33.054981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01dca9-d3af-48e0-b6d7-c1ef58f55f46 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:33.066412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:33.123032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:33.123804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.124156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.124429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.125192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.127118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.157752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1180/2352] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:33 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:33.161928 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:33.175966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a00d6-3856-4b7e-afb0-08a0e6a12f49 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:33.181180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:33.182065 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.182400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.182728 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.183481 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.185177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.199279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1173/2353] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:33.203219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:33.214645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f27ae-982b-486a-bd31-5461f7d21f8c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:33.231861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:33.232397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.232729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.233092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] PATH_INFO: `/v3/projects/1713fbc8595e43a0af186dd1f68c5675/users/ba95bcb9e39740e2b7d9cb9a45a6edbb/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.251188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1713fbc8595e43a0af186dd1f68c5675, user_id=ba95bcb9e39740e2b7d9cb9a45a6edbb, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.254460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.271297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1181/2354] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:33 2022] PUT /identity/v3/projects/1713fbc8595e43a0af186dd1f68c5675/users/ba95bcb9e39740e2b7d9cb9a45a6edbb/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:42:33.275491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:33.288682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbfd63-5e0a-4fda-84c6-ac83893c9fb7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:33.356921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69366fb0-411f-4921-a41f-e791492fe549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:33.358192 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69366fb0-411f-4921-a41f-e791492fe549 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.358422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69366fb0-411f-4921-a41f-e791492fe549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.359066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69366fb0-411f-4921-a41f-e791492fe549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.360126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69366fb0-411f-4921-a41f-e791492fe549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.361824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69366fb0-411f-4921-a41f-e791492fe549 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.376047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1174/2355] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:33.378156 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:33.390343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79f2a3-94d8-493c-afa0-eaf4e0b7217a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:33.395973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:33.396498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.396789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.397077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] PATH_INFO: `/v3/projects/1713fbc8595e43a0af186dd1f68c5675/users/ba95bcb9e39740e2b7d9cb9a45a6edbb/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.399132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1713fbc8595e43a0af186dd1f68c5675, user_id=ba95bcb9e39740e2b7d9cb9a45a6edbb, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.401897 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.413407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1182/2356] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:33 2022] PUT /identity/v3/projects/1713fbc8595e43a0af186dd1f68c5675/users/ba95bcb9e39740e2b7d9cb9a45a6edbb/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:33.423933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.424221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.424569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.474546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] MFA Rules not processed for user `ba95bcb9e39740e2b7d9cb9a45a6edbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:33.475974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:33.566218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1175/2357] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:33.577488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:33.579226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:33.662020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac74f-1cf6-4ac2-b8c5-45cd2cc9fa81 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:33.666131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:33.666892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:33.667241 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:33.667581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:33.739273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:33.740737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:33.751186 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1183/2358] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:35.194541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:35.196457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:35.258794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59eea1-b3cf-477b-af71-47531f680e09 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:35.262443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:35.263070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:35.263341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:35.263606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:35.265434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:35.275020 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:35.276495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:35.281013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:35.285825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1176/2359] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:35.797476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:35.799270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:35.824652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:35.889458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfdd32-6793-4be8-9641-a437acd3fd23 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:35.895588 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:35.896220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:35.896495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:35.896790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:35.898779 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:35.958096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:35.959688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7bd6b-d8e2-4dd0-b5ce-9b98d1fa4031 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:35.969444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1184/2360] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:36.360005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:36.426331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0694d-8946-4a64-b4d4-7ee409051652 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:36.431922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:36.432564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:36.432872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:36.433139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] PATH_INFO: `/v3/users/2cf9c70427b54be39b99b1887a244c54` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:36.434404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf9c70427b54be39b99b1887a244c54)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:36.436339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:36.483261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf9c70427b54be39b99b1887a244c54'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:36.491699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf9c70427b54be39b99b1887a244c54'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:36.492417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf9c70427b54be39b99b1887a244c54'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:36.508558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf9c70427b54be39b99b1887a244c54'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:36.516709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf9c70427b54be39b99b1887a244c54'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:36.523346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1177/2361] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:36 2022] DELETE /identity/v3/users/2cf9c70427b54be39b99b1887a244c54 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:36.528045 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:36.528355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:36.528622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:36.548706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:36.549970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:36.611111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1185/2362] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:36.621446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:36.623204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:36.683889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3d85b-25ad-4146-adf8-d5effc23ea2e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:36.687491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:36.688130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:36.688396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:36.688665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:36.698269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:36.699752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:36.710142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1178/2363] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:36.993061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:36.994549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:37.007718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d282f-e85a-427b-828d-fb55851c95d7 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:37.013094 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:37.013821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.014104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.014380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] PATH_INFO: `/v3/projects/e46acd2afd554f28b528ef26cf02a484` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.016180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46acd2afd554f28b528ef26cf02a484)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:37.018688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:37.073515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46acd2afd554f28b528ef26cf02a484', 'request_id': 'req-8777c5b2-3e7b-4186-9427-e8b890124d68'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.075888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:37.080716 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46acd2afd554f28b528ef26cf02a484', 'request_id': 'req-8777c5b2-3e7b-4186-9427-e8b890124d68'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.087249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46acd2afd554f28b528ef26cf02a484', 'request_id': 'req-8777c5b2-3e7b-4186-9427-e8b890124d68'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.101210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] The token cache is being invalidate because project e46acd2afd554f28b528ef26cf02a484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:37.101492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.102278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.103907 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1186/2364] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:36 2022] DELETE /identity/v3/projects/e46acd2afd554f28b528ef26cf02a484 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:37.108296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:37.118546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086a7a-0f9c-4a1c-8520-8e99b58daa4a service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:37.137869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:37.181392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:37.182050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.182376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.182647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] PATH_INFO: `/v3/users/ba95bcb9e39740e2b7d9cb9a45a6edbb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.183819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba95bcb9e39740e2b7d9cb9a45a6edbb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:37.185804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:37.258064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba95bcb9e39740e2b7d9cb9a45a6edbb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.267585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba95bcb9e39740e2b7d9cb9a45a6edbb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.268242 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba95bcb9e39740e2b7d9cb9a45a6edbb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.280549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba95bcb9e39740e2b7d9cb9a45a6edbb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.287267 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-98ec7222-cb82-495b-b0e9-aad568e668a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba95bcb9e39740e2b7d9cb9a45a6edbb'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.293309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1179/2365] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:42:37 2022] DELETE /identity/v3/users/ba95bcb9e39740e2b7d9cb9a45a6edbb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:37.578155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:37.579769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:37.593921 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777c5b2-3e7b-4186-9427-e8b890124d68 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:37.654041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:37.654688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.654966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.655769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] PATH_INFO: `/v3/projects/1713fbc8595e43a0af186dd1f68c5675` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.657021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1713fbc8595e43a0af186dd1f68c5675)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:37.658944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:37.706973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1713fbc8595e43a0af186dd1f68c5675', 'request_id': 'req-9fac58dc-34d0-4a76-95aa-a09cf34e7755'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.713668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1713fbc8595e43a0af186dd1f68c5675', 'request_id': 'req-9fac58dc-34d0-4a76-95aa-a09cf34e7755'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.720054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1713fbc8595e43a0af186dd1f68c5675', 'request_id': 'req-9fac58dc-34d0-4a76-95aa-a09cf34e7755'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.736151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] The token cache is being invalidate because project 1713fbc8595e43a0af186dd1f68c5675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:42:37.736459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.737154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9fac58dc-34d0-4a76-95aa-a09cf34e7755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:42:37.738765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1187/2366] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:42:37 2022] DELETE /identity/v3/projects/1713fbc8595e43a0af186dd1f68c5675 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:37.744892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.745402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.745661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.766416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:37.767863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:37.835292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1180/2367] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:37.843526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.843861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.844103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.867202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:37.868528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:37.878124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1188/2368] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:37.882311 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:37.892893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740293-1ccc-4d25-8ab4-3bb6988016b8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:37.898451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:37.899296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.899622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.899867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.900587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:42:37.900999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:37.902745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:37.924365 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1181/2369] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:42:37.929279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:37.943296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf6ed4-4000-43d6-b34e-6d6db90dc532 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:37.948303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:37.949036 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:37.949363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:37.949691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:37.950433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:37.952423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:37.986362 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1189/2370] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:42:37 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:37.990542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:38.000848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d0013-91a5-4b0f-a23a-c1e46bad0fd7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:38.027420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:38.065330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:38.065962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:38.066229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:38.066502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:38.067167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:38.069156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:38.091910 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1182/2371] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:42:37 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:42:38.096751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:38.109292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58569450-cebd-486e-8a7a-6c68be14810e admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:38.114952 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:38.115592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:38.115864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:38.116142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:38.116898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:38.118507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:38.129442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1190/2372] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:38.135130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:38.145870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7405d3-4793-43df-b6af-e5a79b05c465 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:38.151002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f08738-f82a-4d1b-9224-1d0d8507953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:38.151622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f08738-f82a-4d1b-9224-1d0d8507953a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:38.151899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f08738-f82a-4d1b-9224-1d0d8507953a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:38.152167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f08738-f82a-4d1b-9224-1d0d8507953a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:38.152912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f08738-f82a-4d1b-9224-1d0d8507953a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:38.154547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f08738-f82a-4d1b-9224-1d0d8507953a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:38.165461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1183/2373] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:42:38.169637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:38.179689 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903c60c-2d5c-4dd2-ac64-4ec5fdb9b171 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:38.184645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:38.185326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:38.185604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:38.185879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] PATH_INFO: `/v3/projects/40d235ddb7124b718656f44dc8109a85/users/3c28c7e6c55d4a2b937a2f2cbff646ae/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:38.202048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d235ddb7124b718656f44dc8109a85, user_id=3c28c7e6c55d4a2b937a2f2cbff646ae, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:38.204742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:38.221347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1191/2374] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:42:38 2022] PUT /identity/v3/projects/40d235ddb7124b718656f44dc8109a85/users/3c28c7e6c55d4a2b937a2f2cbff646ae/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:42:38.295294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a955027-7871-4d4d-895f-43bd97c864c2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:38.295294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a955027-7871-4d4d-895f-43bd97c864c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:38.295294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a955027-7871-4d4d-895f-43bd97c864c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:38.326535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7a955027-7871-4d4d-895f-43bd97c864c2 None None] MFA Rules not processed for user `3c28c7e6c55d4a2b937a2f2cbff646ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:38.327984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7a955027-7871-4d4d-895f-43bd97c864c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:38.399607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1184/2375] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:42:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:38.412857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:38.415319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:38.427706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:38.483396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5a7eb-14d7-414c-a236-b4778406fa04 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:38.487281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81d0ed-51fd-4712-a866-60845f11dea5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:38.487924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81d0ed-51fd-4712-a866-60845f11dea5 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:38.488201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81d0ed-51fd-4712-a866-60845f11dea5 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:38.488469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81d0ed-51fd-4712-a866-60845f11dea5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:38.498015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81d0ed-51fd-4712-a866-60845f11dea5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:38.499522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81d0ed-51fd-4712-a866-60845f11dea5 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:38.509564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1192/2376] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:39.465275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24174ec-2b2b-4c6d-9e08-c66ae45df1d7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.465925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24174ec-2b2b-4c6d-9e08-c66ae45df1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.465925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24174ec-2b2b-4c6d-9e08-c66ae45df1d7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.467355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1185/2377] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:42:39.473757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff761caa-01bc-42fe-8390-4ff975cd6b83 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.474280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff761caa-01bc-42fe-8390-4ff975cd6b83 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.474280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff761caa-01bc-42fe-8390-4ff975cd6b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.497101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-ff761caa-01bc-42fe-8390-4ff975cd6b83 None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:39.498335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ff761caa-01bc-42fe-8390-4ff975cd6b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:39.572373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1193/2378] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:42:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:42:39.584153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24174ec-2b2b-4c6d-9e08-c66ae45df1d7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:39.585855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b24174ec-2b2b-4c6d-9e08-c66ae45df1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:39.609070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24174ec-2b2b-4c6d-9e08-c66ae45df1d7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:39.612639 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2143e-9993-488e-a1f9-bdfd846c688b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:39.613302 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2143e-9993-488e-a1f9-bdfd846c688b service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.613577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2143e-9993-488e-a1f9-bdfd846c688b service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.613845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2143e-9993-488e-a1f9-bdfd846c688b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.624835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2143e-9993-488e-a1f9-bdfd846c688b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:39.626315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2143e-9993-488e-a1f9-bdfd846c688b service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:39.637436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1186/2379] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:42:39.709839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189866f-e025-4dec-bb84-9cca9264c16d None None] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.710133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189866f-e025-4dec-bb84-9cca9264c16d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.710399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189866f-e025-4dec-bb84-9cca9264c16d None None] PATH_INFO: `/` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.711888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1194/2380] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 18:42:39.716804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38bec-fed7-4661-8065-3833c07fb513 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.717142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38bec-fed7-4661-8065-3833c07fb513 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.717415 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38bec-fed7-4661-8065-3833c07fb513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.742287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-64a38bec-fed7-4661-8065-3833c07fb513 None None] MFA Rules not processed for user `f590f668ad724bf087b164bcf3f543ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:39.744271 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-64a38bec-fed7-4661-8065-3833c07fb513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:39.802282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1187/2381] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:42:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:39.814376 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189866f-e025-4dec-bb84-9cca9264c16d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:39.816482 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8189866f-e025-4dec-bb84-9cca9264c16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:39.830954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189866f-e025-4dec-bb84-9cca9264c16d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:39.834926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f591d84-0eac-4198-84aa-65ae9d48d42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:39.835546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591d84-0eac-4198-84aa-65ae9d48d42f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.835815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591d84-0eac-4198-84aa-65ae9d48d42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.836176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591d84-0eac-4198-84aa-65ae9d48d42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.844121 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f591d84-0eac-4198-84aa-65ae9d48d42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:39.845574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f591d84-0eac-4198-84aa-65ae9d48d42f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:39.853455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1195/2382] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:39.941944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef8d3-4a94-4429-ad5d-1b809e776d78 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.942230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef8d3-4a94-4429-ad5d-1b809e776d78 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.942525 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef8d3-4a94-4429-ad5d-1b809e776d78 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.943930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1188/2383] 10.222.0.49 () {54 vars in 921 bytes} [Sun Jul 31 18:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 18:42:39.949312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:39.949592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:39.949859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:39.972638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] MFA Rules not processed for user `9958737194be4488a1e09e6e48557d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:39.973999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:39.985405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1196/2384] 10.222.0.49 () {58 vars in 1018 bytes} [Sun Jul 31 18:42:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:39.996408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef8d3-4a94-4429-ad5d-1b809e776d78 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:40.019402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef8d3-4a94-4429-ad5d-1b809e776d78 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:40.023936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07c427-320a-4e9f-8724-1fa3effb125c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:40.024561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07c427-320a-4e9f-8724-1fa3effb125c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:40.024866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07c427-320a-4e9f-8724-1fa3effb125c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:40.025132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07c427-320a-4e9f-8724-1fa3effb125c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:40.036647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07c427-320a-4e9f-8724-1fa3effb125c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:40.038104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07c427-320a-4e9f-8724-1fa3effb125c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:40.049698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1189/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:40.115935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:40.117648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:40.190662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec08e3-0e8d-42e4-8552-e5f88173ff8d None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:40.197023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:40.197641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:40.197922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:40.198192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:40.200183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:40.212117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:40.213724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:40.224693 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1197/2386] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:41.839377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:41.839858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:41.839950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:41.860076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:42:41.861354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:41.876734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:41.928040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1190/2387] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:42:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:42:41.941526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:41.942919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:41.956080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:42.032414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd990aa-b40d-4d55-a656-278038da2bbd service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:42.038039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:42.038672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:42.038947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:42.039214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:42.048621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:42.050265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:42.058574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1198/2388] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:42:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:42:49.196150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:42:49.197745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:49.235600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f94fc3-f2e8-4acf-ad08-aae3f8436018 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:42:49.241890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:42:49.242547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:42:49.242824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:42:49.243347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:42:49.246054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:42:49.324255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:42:49.326140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:42:49.337280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1191/2389] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:42:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:42:52.055286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=27 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:42:59.332464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=44 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:00.099515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:00.099515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:00.115140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f315-1bf5-4412-be6a-a16d705172a2 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:00.121790 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:00.122572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:00.123203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:00.123480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:00.125320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:00.142799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:00.145185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f1907-b87a-4210-9ba8-9db930be8bed service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:00.157028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1199/2390] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:04.465764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:04.529085 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5cd5e-4cff-41b0-bf14-87a675b827e3 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:04.534462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:04.535098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:04.535377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:04.535645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] PATH_INFO: `/v3/users/3c28c7e6c55d4a2b937a2f2cbff646ae` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:04.536879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c28c7e6c55d4a2b937a2f2cbff646ae)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:04.538815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:04.597931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c28c7e6c55d4a2b937a2f2cbff646ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:04.608061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c28c7e6c55d4a2b937a2f2cbff646ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:04.608746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c28c7e6c55d4a2b937a2f2cbff646ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:04.626813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c28c7e6c55d4a2b937a2f2cbff646ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:04.638334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c28c7e6c55d4a2b937a2f2cbff646ae'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:04.644974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1192/2391] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:43:04 2022] DELETE /identity/v3/users/3c28c7e6c55d4a2b937a2f2cbff646ae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:04.649697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:04.650216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:04.650216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:04.670275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:04.671589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:04.733550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1200/2392] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:43:04.743565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:04.744991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:04.807098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ab3c0-8bd9-4f97-91b4-1a5cb8188de0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:04.811142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b83ba4-e544-4d7a-9aa6-3a8e3bb8ba3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:04.811851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b83ba4-e544-4d7a-9aa6-3a8e3bb8ba3c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:04.812269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b83ba4-e544-4d7a-9aa6-3a8e3bb8ba3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:04.812718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b83ba4-e544-4d7a-9aa6-3a8e3bb8ba3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:04.822285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b83ba4-e544-4d7a-9aa6-3a8e3bb8ba3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:04.823945 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b83ba4-e544-4d7a-9aa6-3a8e3bb8ba3c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:04.833631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1193/2393] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:05.113359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.114781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:05.127264 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e41a6f-2dd0-4719-a6f0-0104f70e8113 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.134616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.135329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.135531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.135784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] PATH_INFO: `/v3/projects/40d235ddb7124b718656f44dc8109a85` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.138024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d235ddb7124b718656f44dc8109a85)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.140584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.195624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d235ddb7124b718656f44dc8109a85', 'request_id': 'req-b01f02b1-47db-4158-ac59-f812268555fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:05.202646 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d235ddb7124b718656f44dc8109a85', 'request_id': 'req-b01f02b1-47db-4158-ac59-f812268555fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:05.209169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d235ddb7124b718656f44dc8109a85', 'request_id': 'req-b01f02b1-47db-4158-ac59-f812268555fd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:05.222854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] The token cache is being invalidate because project 40d235ddb7124b718656f44dc8109a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:43:05.223140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:05.223753 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-b01f02b1-47db-4158-ac59-f812268555fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:05.225329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1201/2394] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:43:05 2022] DELETE /identity/v3/projects/40d235ddb7124b718656f44dc8109a85 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:05.230473 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.230752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.231022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.251567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:05.253113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:05.271448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:05.319626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1194/2395] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:05.327711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.327989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.328260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.348724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:05.349978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:05.358730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1202/2396] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:05.363027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.372997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49710073-abd1-4a58-bff6-aa1dd855d49a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.378432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.379045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.379318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.379582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.380266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:43:05.380565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.382221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.393851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1195/2397] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:43:05.398323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.407723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa332-442b-4aa5-95d0-737d0b6c6030 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.412859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.413469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.413757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.414020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.414677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.416585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.446815 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1203/2398] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:05 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:05.451414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.461681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db4d01-a644-42a2-8898-2fae93c3d994 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.520912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.521608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.521793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.522094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.522847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.524801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.547189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1196/2399] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:05 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:05.551684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.563225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b551fe-ca13-4b7e-9cde-fa94d3f91a70 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.568724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.569361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.569694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.569895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.570578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.572179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.582446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1204/2400] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:05.586687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.597565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a16aed-a58d-444f-9b9c-b19ba6f69d34 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.603480 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec699945-a1f3-4ea8-b1a8-4c239a05a732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.604283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec699945-a1f3-4ea8-b1a8-4c239a05a732 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.604602 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec699945-a1f3-4ea8-b1a8-4c239a05a732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.604903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec699945-a1f3-4ea8-b1a8-4c239a05a732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.605593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec699945-a1f3-4ea8-b1a8-4c239a05a732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.607320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec699945-a1f3-4ea8-b1a8-4c239a05a732 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.617266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1197/2401] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:05.622102 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.633228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d832e95-2826-4c8c-84b3-4c340d5d7f2b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.638516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.639162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.639432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.639700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] PATH_INFO: `/v3/projects/9540ab0260cc4c0ca76036b79bb5f506/users/f5d3517f51b74230964f1e2b29371e0b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.656396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9540ab0260cc4c0ca76036b79bb5f506, user_id=f5d3517f51b74230964f1e2b29371e0b, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.659104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.674891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1205/2402] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:05 2022] PUT /identity/v3/projects/9540ab0260cc4c0ca76036b79bb5f506/users/f5d3517f51b74230964f1e2b29371e0b/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:43:05.680025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c745e5b-d555-4544-bc86-5afc0161ee14 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.680295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c745e5b-d555-4544-bc86-5afc0161ee14 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.680561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c745e5b-d555-4544-bc86-5afc0161ee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.701964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6c745e5b-d555-4544-bc86-5afc0161ee14 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:05.703215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6c745e5b-d555-4544-bc86-5afc0161ee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:05.779668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1198/2403] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:05.789688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:05.791714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:05.802830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:05.856579 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28de27e-e857-4348-a761-003cbbb4f672 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:05.860925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:05.861700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:05.861982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:05.862319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:05.872959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:05.874406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:05.884313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1206/2404] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:06.526855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5292-d28a-496a-95c8-649b9c5064e2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:06.527381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5292-d28a-496a-95c8-649b9c5064e2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:06.527381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5292-d28a-496a-95c8-649b9c5064e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:06.556740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-24ec5292-d28a-496a-95c8-649b9c5064e2 None None] MFA Rules not processed for user `f5d3517f51b74230964f1e2b29371e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:06.557998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-24ec5292-d28a-496a-95c8-649b9c5064e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:06.576951 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:06.633104 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1199/2405] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:06.648034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:06.649526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:06.709256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6747-b2b1-4607-8ac2-080b1fabc7f7 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:06.712924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:06.713548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:06.713824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:06.714093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:06.722918 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:06.724380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:06.733648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1207/2406] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:08.095207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695829c5-a108-411b-b6d5-17dab9046dc1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:08.095733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695829c5-a108-411b-b6d5-17dab9046dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:08.095733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695829c5-a108-411b-b6d5-17dab9046dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:08.117041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-695829c5-a108-411b-b6d5-17dab9046dc1 None None] MFA Rules not processed for user `f5d3517f51b74230964f1e2b29371e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:08.118239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-695829c5-a108-411b-b6d5-17dab9046dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:08.132298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1200/2407] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:08.372848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:08.374430 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:08.387536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57669b03-9d9b-4670-bf11-0ba1ff680609 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:08.391053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:08.391682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:08.391953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:08.392216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:08.394030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:08.403782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:08.405227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:08.413514 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1208/2408] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:12.133169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:12.134001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:12.134280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:12.155567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:12.156811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:12.218753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1201/2409] 10.222.0.49 () {58 vars in 967 bytes} [Sun Jul 31 18:43:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:12.231681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:12.233165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:12.302447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11460516-25da-436f-a370-12697211c3a5 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:12.307686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:12.308305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:12.308582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:12.308890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:12.317868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:12.319456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:12.327706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1209/2410] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:43:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:43:12.699618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:12.701292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:12.790418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c7edc-894e-4a07-8916-f0221edd421f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:12.796506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:12.797168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:12.797474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:12.797745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:12.799485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:12.811733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:12.813296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:12.824181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1202/2411] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:22.324244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=87 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:22.820376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:28.548829 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:28.560871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146d4c2-7fc8-487b-8a71-a45d5048dc92 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:28.565187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:28.566890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:28.567216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:28.567493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:28.570133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:28.586402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:28.588016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:28.598904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1210/2412] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:32.038110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.039653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:32.126266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.128312 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:32.143003 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea8b18-c883-4186-b3e2-f4789c62e461 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.148489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee420b-76e4-4d61-8fcd-6b0ac07c34ff service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.149129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.149767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.150204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.150652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.152511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:32.154649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.155639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.155977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.156345 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] PATH_INFO: `/v3/users/88472fa130654657af599106c6aac279` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.157533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88472fa130654657af599106c6aac279)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.159435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:32.166554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.168667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:32.180810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1203/2413] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:32.226118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88472fa130654657af599106c6aac279'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.235108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88472fa130654657af599106c6aac279'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.235712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88472fa130654657af599106c6aac279'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.243717 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88472fa130654657af599106c6aac279'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.244414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-649d2156-b403-4c64-8247-39fefb68d535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88472fa130654657af599106c6aac279'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.263632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1211/2414] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:43:32 2022] DELETE /identity/v3/users/88472fa130654657af599106c6aac279 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:32.457724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.475217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94767-16b7-4af8-be20-582cb0593a44 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.543646 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.544293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.544558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.544848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] PATH_INFO: `/v3/projects/8b2df629457c4ff68ad061eb29cb232f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.546016 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2df629457c4ff68ad061eb29cb232f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.547939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:32.597064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2df629457c4ff68ad061eb29cb232f', 'request_id': 'req-346f26aa-0c48-480c-bfae-5d362a739eda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.604597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2df629457c4ff68ad061eb29cb232f', 'request_id': 'req-346f26aa-0c48-480c-bfae-5d362a739eda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.613732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2df629457c4ff68ad061eb29cb232f', 'request_id': 'req-346f26aa-0c48-480c-bfae-5d362a739eda'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.631460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] The token cache is being invalidate because project 8b2df629457c4ff68ad061eb29cb232f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:43:32.631744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.632446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-346f26aa-0c48-480c-bfae-5d362a739eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:32.634061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1204/2415] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:43:32 2022] DELETE /identity/v3/projects/8b2df629457c4ff68ad061eb29cb232f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:32.639187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.639556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.639896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.662115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:32.663415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:32.730741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1212/2416] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:32.739649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.739932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.740201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.762700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:32.763906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:32.772159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1205/2417] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:32.776540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.786692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3598f6-a769-4b4e-aaa1-dc3af01f91c6 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.792796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.793437 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.793709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.793973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.794777 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:43:32.795073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.796682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:32.809429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1213/2418] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:43:32.814025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.823432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef056c40-2b25-4ecf-b848-68043da4907c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.829743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.830400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.830665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.830930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.831597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.833523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:32.845843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:32.865848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1206/2419] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:32 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:32.870294 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.880751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ae74-0b19-435b-a666-461e79bc2853 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.926464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:32.940874 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.941504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.941795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.942048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.942741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.944662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:32.965599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1214/2420] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:32 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:32.969865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:32.980942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac588333-d460-4e26-9080-5ef5d615654a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:32.986254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:32.986612 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:32.986885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:32.987492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:32.988219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:32.989859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.001947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1207/2421] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:33.006494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.022540 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91e037-3818-445f-a545-00b595092967 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.029415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.029671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.029987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.030324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.031328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.033575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.045865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1215/2422] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:33.050142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.065146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def3107-5c0e-4b68-9ef1-bb88f6987f8a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.065744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.067600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:33.069799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c17365-467c-48a9-9ab5-be6fb109eaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.070471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c17365-467c-48a9-9ab5-be6fb109eaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.070761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c17365-467c-48a9-9ab5-be6fb109eaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.071044 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c17365-467c-48a9-9ab5-be6fb109eaf6 admin admin] PATH_INFO: `/v3/projects/52f5c75ba0e144728a118db8ecf63cf4/users/7f7f0d28f8874bdfa9e2f726c9ae62ec/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.090924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c17365-467c-48a9-9ab5-be6fb109eaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f5c75ba0e144728a118db8ecf63cf4, user_id=7f7f0d28f8874bdfa9e2f726c9ae62ec, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.093712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c17365-467c-48a9-9ab5-be6fb109eaf6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.109847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1208/2423] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:33 2022] PUT /identity/v3/projects/52f5c75ba0e144728a118db8ecf63cf4/users/7f7f0d28f8874bdfa9e2f726c9ae62ec/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:33.120198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.120475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.120814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.129523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaefdc-0804-40c7-bb5d-35c17f69c927 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.155812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] MFA Rules not processed for user `7f7f0d28f8874bdfa9e2f726c9ae62ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:33.157300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:33.188436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.189089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.189366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.189637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.191402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:33.230026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1209/2424] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:43:33.234666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.260131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.261755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.271583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1216/2425] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:33.295796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad37e9-4824-472b-967b-dd5d8e0066e8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.300661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.301462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.301739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.302002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.302694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.304574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.334724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1210/2426] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:33 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:43:33.338806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.348854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743240ec-13af-48ed-aba2-6881a35d11d8 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.379981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:33.412500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.413015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.413290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.413560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.414239 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.416269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.439487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1217/2427] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:33 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:33.443736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.455037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021672a-8a8c-44e2-970b-87019a70fdfe admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.460014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.460692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.461023 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.461297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.462004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.463621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.474987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1211/2428] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:33.479268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.493699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329323b-f8df-4368-b0b2-98ae443bb65a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.500692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.501421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.501784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.501946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] PATH_INFO: `/v3/projects/1de7da55d0904421bd38884790dc90b5/users/0a454768f19c4b2aaf4f98b9048adfc0/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.521471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de7da55d0904421bd38884790dc90b5, user_id=0a454768f19c4b2aaf4f98b9048adfc0, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.525056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.538744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1218/2429] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:33 2022] PUT /identity/v3/projects/1de7da55d0904421bd38884790dc90b5/users/0a454768f19c4b2aaf4f98b9048adfc0/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:33.542954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.553747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b5e5a-7a43-4c2d-9923-f03cf470a1ae admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.570877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:33.614690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40bbc-b462-434b-8e16-dbf1b678d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.615308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40bbc-b462-434b-8e16-dbf1b678d4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.615580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40bbc-b462-434b-8e16-dbf1b678d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.615862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40bbc-b462-434b-8e16-dbf1b678d4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.616532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40bbc-b462-434b-8e16-dbf1b678d4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.618164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d40bbc-b462-434b-8e16-dbf1b678d4c3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.628328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1212/2430] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:33.632782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.644529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12530-2e7f-4bd5-ad49-3790c2ac6a17 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.650460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.651138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.651424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.651688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] PATH_INFO: `/v3/projects/1de7da55d0904421bd38884790dc90b5/users/0a454768f19c4b2aaf4f98b9048adfc0/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.653706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de7da55d0904421bd38884790dc90b5, user_id=0a454768f19c4b2aaf4f98b9048adfc0, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.656392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.667157 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1219/2431] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:33 2022] PUT /identity/v3/projects/1de7da55d0904421bd38884790dc90b5/users/0a454768f19c4b2aaf4f98b9048adfc0/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:33.679073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.679377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.679648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.708577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] MFA Rules not processed for user `0a454768f19c4b2aaf4f98b9048adfc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:33.709907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:33.784261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1213/2432] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:33.796258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.798261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:33.859143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4165ffe-64e7-4ddf-8dc3-812625bd4867 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.862811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.863432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.863704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.863970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:33.873235 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:33.874660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:33.884556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1220/2433] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:33.961215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:33.971050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7a987-16dc-4a75-afd4-5a16ffa3d6b7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:33.974994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:33.975617 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:33.975893 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:33.976160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:34.040956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:34.042694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:34.052293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1214/2434] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:43:36.404687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:36.406303 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:36.461905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:36.463423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:36.483083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadeeb2-c5e2-4fe7-b2d9-09cc6de0ced0 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:36.488627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989feca9-3589-4258-8536-32f5083f105f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:36.489453 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989feca9-3589-4258-8536-32f5083f105f service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:36.489724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989feca9-3589-4258-8536-32f5083f105f service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:36.490018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989feca9-3589-4258-8536-32f5083f105f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:36.491770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-989feca9-3589-4258-8536-32f5083f105f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:36.503930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989feca9-3589-4258-8536-32f5083f105f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:36.505544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989feca9-3589-4258-8536-32f5083f105f service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:36.509770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:36.515796 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1221/2435] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:36.534028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd56c87-a02a-488e-97aa-a698635ec4cc service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:36.538322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:36.539027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:36.539354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:36.539626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] PATH_INFO: `/v3/users/4dc88d9b0bc743119dcb7dc2fa8fdcd6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:36.540800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc88d9b0bc743119dcb7dc2fa8fdcd6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:36.542805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:36.592828 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc88d9b0bc743119dcb7dc2fa8fdcd6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.601765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc88d9b0bc743119dcb7dc2fa8fdcd6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.602417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc88d9b0bc743119dcb7dc2fa8fdcd6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.615373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc88d9b0bc743119dcb7dc2fa8fdcd6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.623938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-30cd1ec2-2e1e-4fb8-bf4d-8e8e3580c805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc88d9b0bc743119dcb7dc2fa8fdcd6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.631626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1215/2436] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:43:36 2022] DELETE /identity/v3/users/4dc88d9b0bc743119dcb7dc2fa8fdcd6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:36.815348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989feca9-3589-4258-8536-32f5083f105f service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:36.837498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989feca9-3589-4258-8536-32f5083f105f service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:36.853835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:36.915809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:36.916985 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:36.918004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:36.918392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] PATH_INFO: `/v3/projects/fc4113d291354d99b2591289639ec248` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:36.919960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4113d291354d99b2591289639ec248)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:36.922043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:36.981350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4113d291354d99b2591289639ec248', 'request_id': 'req-3781e5a2-97c0-4e62-93af-93703d376574'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.988799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4113d291354d99b2591289639ec248', 'request_id': 'req-3781e5a2-97c0-4e62-93af-93703d376574'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:36.994475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4113d291354d99b2591289639ec248', 'request_id': 'req-3781e5a2-97c0-4e62-93af-93703d376574'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:37.007957 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] The token cache is being invalidate because project fc4113d291354d99b2591289639ec248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:43:37.008242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:37.008929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-3781e5a2-97c0-4e62-93af-93703d376574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:37.010835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1222/2437] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:43:36 2022] DELETE /identity/v3/projects/fc4113d291354d99b2591289639ec248 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:37.015603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.015877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.016149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.038169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:37.039289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:37.098370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1216/2438] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:37.106383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.106704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.106932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.127762 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:37.129061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:37.137098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1223/2439] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:37.141419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.151682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe70f4c-cf77-4caf-84cc-091076d794bf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.156680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.157350 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.157623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.157897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.158659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:43:37.158958 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.160562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.174629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1217/2440] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:43:37.179272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.190894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367fc03-eb06-4443-b065-6837815ddfda None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.198211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.199296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.199792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.200279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.201166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.203071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.244068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1224/2441] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:43:37.248713 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.260486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb389a-6140-434c-93a2-08a4e2b5a9b8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.306593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:37.322325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.322948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.323227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.323498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.324180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.326231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.352196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1218/2442] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:43:37.354639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.365132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a3253-57ce-4067-a7bb-c4946f88e3bb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.370381 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.371125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.371473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.371747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.372463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.374147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.384938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1225/2443] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:37.389082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.399564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4490929-e9b0-4df9-b27a-cd11e8a8ac55 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.405140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095871d-75fc-42bc-95fd-d2e241c559f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.405762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095871d-75fc-42bc-95fd-d2e241c559f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.406036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095871d-75fc-42bc-95fd-d2e241c559f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.406304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095871d-75fc-42bc-95fd-d2e241c559f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.406985 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095871d-75fc-42bc-95fd-d2e241c559f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.408582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095871d-75fc-42bc-95fd-d2e241c559f3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.419841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1219/2444] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:37.424162 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.434707 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb6820-7dff-46ad-a449-a990d99a4384 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.440268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.440987 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.441270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.441539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] PATH_INFO: `/v3/projects/db88156d85df41beb0e2276cac9c4dd4/users/a6c14e93f2074723921836f0635b0ae2/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.457808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db88156d85df41beb0e2276cac9c4dd4, user_id=a6c14e93f2074723921836f0635b0ae2, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.460510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.477151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1226/2445] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:37 2022] PUT /identity/v3/projects/db88156d85df41beb0e2276cac9c4dd4/users/a6c14e93f2074723921836f0635b0ae2/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:43:37.489920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.490419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.490530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.527178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] MFA Rules not processed for user `a6c14e93f2074723921836f0635b0ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:37.528324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:37.595855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1220/2446] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:37.601166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.650937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:37.671915 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b68d-11fc-415a-98c2-a77446f27544 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.677064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.677677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.677948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.678212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.678861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.680833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.709182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1227/2447] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:37.713380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.724817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9d7a-0041-4d32-a5e1-41918eba1f6e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.782811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.783353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.783631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.783934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.784586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.786582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.807265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1221/2448] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:37 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:37.811433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.821894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3bd5-d84b-4bcd-8f1e-ee2f2ce468cc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.827323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.827922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.828205 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.828454 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.829168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.830764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.841760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1228/2449] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:37.845914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.856414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733982a-3264-447c-87a3-23486eedea16 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.861281 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.861936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.862212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.862497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] PATH_INFO: `/v3/projects/4419ff675d1a4accb65da8fbeaa105f6/users/90321f4b767f46a6b8003eb8ff9b6f2c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.877865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4419ff675d1a4accb65da8fbeaa105f6, user_id=90321f4b767f46a6b8003eb8ff9b6f2c, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.880574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.895091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1222/2450] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:37 2022] PUT /identity/v3/projects/4419ff675d1a4accb65da8fbeaa105f6/users/90321f4b767f46a6b8003eb8ff9b6f2c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:37.899358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:37.909656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7dcda-1bfd-4376-a055-d6e163fa9ab4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:37.972458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17df26-ca22-4a47-b7fe-f42e4e55054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:37.973243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17df26-ca22-4a47-b7fe-f42e4e55054c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:37.973406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17df26-ca22-4a47-b7fe-f42e4e55054c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:37.973667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17df26-ca22-4a47-b7fe-f42e4e55054c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:37.974369 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17df26-ca22-4a47-b7fe-f42e4e55054c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:37.976101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17df26-ca22-4a47-b7fe-f42e4e55054c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:37.986908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1229/2451] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:37.991127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:38.001862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801908b-8628-4f7f-93bb-c52944d25a1f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:38.007083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:38.007761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:38.008036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:38.008308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] PATH_INFO: `/v3/projects/4419ff675d1a4accb65da8fbeaa105f6/users/90321f4b767f46a6b8003eb8ff9b6f2c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:38.010397 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4419ff675d1a4accb65da8fbeaa105f6, user_id=90321f4b767f46a6b8003eb8ff9b6f2c, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:38.013309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:38.029465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1223/2452] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:37 2022] PUT /identity/v3/projects/4419ff675d1a4accb65da8fbeaa105f6/users/90321f4b767f46a6b8003eb8ff9b6f2c/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:38.040666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:38.041077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:38.041414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:38.070657 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] MFA Rules not processed for user `90321f4b767f46a6b8003eb8ff9b6f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:38.071936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:38.143856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1230/2453] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:43:38.155696 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:38.157834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:38.168248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:38.217252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97807219-6eb9-4eae-9bbb-2a8f5d522207 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:38.221629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:38.222328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:38.222670 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:38.222999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:38.233136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:38.234575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:38.244173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1224/2454] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:38.322305 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:38.332694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca6420-220e-4784-8bf7-d9f04b9ab337 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:38.336862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c7fd1-45b5-437a-91d2-b1b1ea88a2ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:38.337467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c7fd1-45b5-437a-91d2-b1b1ea88a2ad service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:38.337736 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c7fd1-45b5-437a-91d2-b1b1ea88a2ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:38.337998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c7fd1-45b5-437a-91d2-b1b1ea88a2ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:38.374261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:38.399455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c7fd1-45b5-437a-91d2-b1b1ea88a2ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:38.401005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c7fd1-45b5-437a-91d2-b1b1ea88a2ad service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:38.410488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1231/2455] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:48.239841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=23 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:48.406064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=11 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:49.428314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:49.493416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01174c-70ee-4dc2-a9a8-757dc125c5f6 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:49.498248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:49.498917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:49.499196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:49.499467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] PATH_INFO: `/v3/users/a6c14e93f2074723921836f0635b0ae2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:49.500653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c14e93f2074723921836f0635b0ae2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:49.503014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:49.552252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c14e93f2074723921836f0635b0ae2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:49.560214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c14e93f2074723921836f0635b0ae2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:49.561317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c14e93f2074723921836f0635b0ae2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:49.575547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c14e93f2074723921836f0635b0ae2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:49.582879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c14e93f2074723921836f0635b0ae2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:49.589025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1225/2456] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:43:49 2022] DELETE /identity/v3/users/a6c14e93f2074723921836f0635b0ae2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:49.593745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:49.594206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:49.594313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:49.617332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:49.618556 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:49.680637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1232/2457] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:49.690538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:49.692096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:49.749309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6a920-cd45-4eec-87d7-e1f73ea6afc1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:49.754050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:49.754632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:49.754902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:49.755161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:49.765239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:49.766676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:49.776278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1226/2458] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:50.059519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:50.061059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:50.078452 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5313-6463-4903-8eda-3f53b01a19c3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:50.083361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:50.084014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:50.084290 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:50.084553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] PATH_INFO: `/v3/projects/db88156d85df41beb0e2276cac9c4dd4` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:50.085784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db88156d85df41beb0e2276cac9c4dd4)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:50.087668 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:50.137909 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db88156d85df41beb0e2276cac9c4dd4', 'request_id': 'req-c492ae6e-c547-4e37-9223-c5c2d5123e67'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.144075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db88156d85df41beb0e2276cac9c4dd4', 'request_id': 'req-c492ae6e-c547-4e37-9223-c5c2d5123e67'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.150914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db88156d85df41beb0e2276cac9c4dd4', 'request_id': 'req-c492ae6e-c547-4e37-9223-c5c2d5123e67'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.166498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] The token cache is being invalidate because project db88156d85df41beb0e2276cac9c4dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:43:50.166782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.167510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.169203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1233/2459] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:43:50 2022] DELETE /identity/v3/projects/db88156d85df41beb0e2276cac9c4dd4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:50.174990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:50.184747 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd70e0c-fe0a-45cb-9385-c11a1c6fb772 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:50.203479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:50.249127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:50.249851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:50.250228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:50.250651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] PATH_INFO: `/v3/users/90321f4b767f46a6b8003eb8ff9b6f2c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:50.252069 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90321f4b767f46a6b8003eb8ff9b6f2c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:50.254368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:50.303426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90321f4b767f46a6b8003eb8ff9b6f2c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.311384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90321f4b767f46a6b8003eb8ff9b6f2c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.312426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90321f4b767f46a6b8003eb8ff9b6f2c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.324556 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90321f4b767f46a6b8003eb8ff9b6f2c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.331743 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24568e38-cc9b-43f4-80fb-8a3eb7d798f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90321f4b767f46a6b8003eb8ff9b6f2c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.337965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1227/2460] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:43:50 2022] DELETE /identity/v3/users/90321f4b767f46a6b8003eb8ff9b6f2c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:50.636080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:50.637740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:50.658666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492ae6e-c547-4e37-9223-c5c2d5123e67 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:50.717908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:50.718459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:50.718738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:50.719009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] PATH_INFO: `/v3/projects/4419ff675d1a4accb65da8fbeaa105f6` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:50.720229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4419ff675d1a4accb65da8fbeaa105f6)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:50.722200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:50.755304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:50.774489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4419ff675d1a4accb65da8fbeaa105f6', 'request_id': 'req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.781574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4419ff675d1a4accb65da8fbeaa105f6', 'request_id': 'req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.787837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4419ff675d1a4accb65da8fbeaa105f6', 'request_id': 'req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.804697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] The token cache is being invalidate because project 4419ff675d1a4accb65da8fbeaa105f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:43:50.805016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.805704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-dc0353ff-40d6-4b0a-8a09-2c2825e5395f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:43:50.807313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1234/2461] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:43:50 2022] DELETE /identity/v3/projects/4419ff675d1a4accb65da8fbeaa105f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:50.813637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:50.814181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:50.814181 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:50.836932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:50.838124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:50.901174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1228/2462] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:50.910756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:50.911034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:50.911302 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:50.931935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:50.933182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:50.941507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1235/2463] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:50.945901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:50.956409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca91970-89cd-4dea-9b2e-8a693505f6d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:50.962876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:50.963494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:50.963771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:50.964041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:50.964742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:43:50.965081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:50.966695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:50.979850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1229/2464] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:43:50.984188 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:50.994021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d17a0-1dc3-4f87-a4c6-4d90f4909361 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:50.999599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.000201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.000474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.000732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.001432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.003662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.036180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1236/2465] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:50 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:51.040994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.054098 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a480877-efee-4b86-ba62-390440f3e7a1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.081461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:51.113590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.114200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.114477 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.114746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.115406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.117401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.143321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1230/2466] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:51 2022] POST /identity/v3/users => generated 279 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:43:51.147685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.158928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be5df8-fdf9-4cb6-8408-d4870e002903 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.164633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.165269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.165537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.165800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.166492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.168077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.178574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1237/2467] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:51.183091 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.193907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7038b-ef87-4601-ba8e-565a185dcf0e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.198962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e9741-3a97-48fa-ae04-14dc911c595c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.199570 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e9741-3a97-48fa-ae04-14dc911c595c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.199847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e9741-3a97-48fa-ae04-14dc911c595c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.200116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e9741-3a97-48fa-ae04-14dc911c595c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.200822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e9741-3a97-48fa-ae04-14dc911c595c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.202444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e9741-3a97-48fa-ae04-14dc911c595c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.213191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1231/2468] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:51.217418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.228197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7c67f-c434-4f22-983a-08bdaaf0ab78 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.233080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.233673 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.233947 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.234212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] PATH_INFO: `/v3/projects/fc9c81c732f14d11b694e0962a459e43/users/1f8146f1a5d34974b24873b9c9f577c2/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.248859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9c81c732f14d11b694e0962a459e43, user_id=1f8146f1a5d34974b24873b9c9f577c2, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.251599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.266523 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1238/2469] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:51 2022] PUT /identity/v3/projects/fc9c81c732f14d11b694e0962a459e43/users/1f8146f1a5d34974b24873b9c9f577c2/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:51.277009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.277300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.277569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.307825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] MFA Rules not processed for user `1f8146f1a5d34974b24873b9c9f577c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:51.309538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:51.376510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1232/2470] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:51 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:51.381872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.446651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff3035-5f74-4a41-9937-a447528b4c1c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.451727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.452348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.452621 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.452917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.453607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.455617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.489309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1239/2471] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:43:51 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:51.493400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.504723 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6e8cc-56a4-40fe-9a4d-6f3ffc7d8236 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.563972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.564610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.564917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.565194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.565849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.567819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.592871 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1233/2472] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:43:51 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:43:51.595158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.605812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9da1e-6a83-49e4-8265-ddf1f562b0df admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.611119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.611730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.612002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.612266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.612993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.614587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.625906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1240/2473] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:43:51.631074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.642094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8818c3d-bc11-4469-a15a-7836c979875f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.648116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.649204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.649488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.649874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] PATH_INFO: `/v3/projects/07afb0f1b47e46ae9625eac4f147c13d/users/6b328595daaa4c919b604ab223bada59/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.666873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07afb0f1b47e46ae9625eac4f147c13d, user_id=6b328595daaa4c919b604ab223bada59, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.669968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.687436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1234/2474] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:51 2022] PUT /identity/v3/projects/07afb0f1b47e46ae9625eac4f147c13d/users/6b328595daaa4c919b604ab223bada59/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:51.691300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.701798 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f0ca3-49bd-4ab3-8f6d-dd5e735e5626 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.723928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:51.771530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2011c7-bef1-47fc-86df-08c85af1e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.772448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2011c7-bef1-47fc-86df-08c85af1e5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.772878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2011c7-bef1-47fc-86df-08c85af1e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.773212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2011c7-bef1-47fc-86df-08c85af1e5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.773965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2011c7-bef1-47fc-86df-08c85af1e5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.775662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2011c7-bef1-47fc-86df-08c85af1e5e9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.786723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1241/2475] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:43:51.791189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.797227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:43:51.805347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0574b-966c-454d-bf21-a70f803ebf0d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:51.811682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:51.812382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.812661 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.812942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] PATH_INFO: `/v3/projects/07afb0f1b47e46ae9625eac4f147c13d/users/6b328595daaa4c919b604ab223bada59/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.815467 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07afb0f1b47e46ae9625eac4f147c13d, user_id=6b328595daaa4c919b604ab223bada59, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:51.818510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:51.839322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1235/2476] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:43:51 2022] PUT /identity/v3/projects/07afb0f1b47e46ae9625eac4f147c13d/users/6b328595daaa4c919b604ab223bada59/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:43:51.841893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:51.842392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:51.842857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:51.878396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] MFA Rules not processed for user `6b328595daaa4c919b604ab223bada59`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:43:51.879663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:51.965487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1242/2477] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:43:51 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:43:51.982317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:51.984343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:43:52.070485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9aa9a-2c23-4910-bb14-654fdfa754b3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:52.078610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:52.079358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:52.079636 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:52.079900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:52.090026 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:52.091496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:52.101469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1236/2478] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:51 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:43:52.184738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:43:52.194603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c40661-2124-44bb-96f0-722bf584f6e5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:43:52.198479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:43:52.199082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:43:52.199351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:43:52.199613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:43:52.265509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:43:52.266914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:43:52.275927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1243/2479] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:43:52 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:02.097183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:02.271859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=80 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:18.506412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:18.507844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:18.579582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f04fc-a3bf-4f25-b062-1d5ae43a0c29 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:18.580090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:18.580940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:18.581298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:18.581658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:18.584012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:18.657831 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:18.659526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5093a-78dc-4258-95e2-86a63ad0f68e service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:18.670757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1237/2480] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:19.181044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:19.192641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28480950-f18f-41f4-be06-cb1f8ad6aea6 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:19.198130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:19.198770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:19.199042 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:19.199315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] PATH_INFO: `/v3/users/1f8146f1a5d34974b24873b9c9f577c2` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:19.200558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f8146f1a5d34974b24873b9c9f577c2)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:19.202516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:19.261130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f8146f1a5d34974b24873b9c9f577c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.269757 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f8146f1a5d34974b24873b9c9f577c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.270373 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f8146f1a5d34974b24873b9c9f577c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.277931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f8146f1a5d34974b24873b9c9f577c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.278682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f8146f1a5d34974b24873b9c9f577c2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.296513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1244/2481] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:19 2022] DELETE /identity/v3/users/1f8146f1a5d34974b24873b9c9f577c2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:19.301340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:19.301620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:19.301891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:19.322295 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:19.323591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:19.384628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1238/2482] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:19.394887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:19.396356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:19.458557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e42551-133c-40c8-9ef3-8544af2390b7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:19.462491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:19.463151 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:19.463405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:19.463674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:19.473759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:19.475202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:19.485361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1245/2483] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:44:19.760944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:19.762466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:19.795787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d220713-da01-42b0-82ae-f395490f27a0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:19.802371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:19.803297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:19.803774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:19.804087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] PATH_INFO: `/v3/projects/fc9c81c732f14d11b694e0962a459e43` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:19.805628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9c81c732f14d11b694e0962a459e43)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:19.811159 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:19.885863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:19.890174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9c81c732f14d11b694e0962a459e43', 'request_id': 'req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.898543 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9c81c732f14d11b694e0962a459e43', 'request_id': 'req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.905861 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9c81c732f14d11b694e0962a459e43', 'request_id': 'req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.922304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] The token cache is being invalidate because project fc9c81c732f14d11b694e0962a459e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:19.922745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.923625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:19.928563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1239/2484] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:19 2022] DELETE /identity/v3/projects/fc9c81c732f14d11b694e0962a459e43 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:19.929886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:19.939986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51f79-f5b8-4668-90bc-c09663806b64 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:20.013064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:20.014425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.014425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.014425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] PATH_INFO: `/v3/users/6b328595daaa4c919b604ab223bada59` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.015933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b328595daaa4c919b604ab223bada59)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:20.018246 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:20.040806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:20.104197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b328595daaa4c919b604ab223bada59'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.112154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b328595daaa4c919b604ab223bada59'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.113734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b328595daaa4c919b604ab223bada59'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.124548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b328595daaa4c919b604ab223bada59'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.126155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-e293e713-9016-4b97-996d-477a1437e4c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b328595daaa4c919b604ab223bada59'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.145911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1246/2485] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:19 2022] DELETE /identity/v3/users/6b328595daaa4c919b604ab223bada59 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:20.437876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:20.439376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:20.457244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72d5d-ec15-4f05-9f7a-1375a721f6bd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:20.527237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:20.528039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.528329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.528603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] PATH_INFO: `/v3/projects/07afb0f1b47e46ae9625eac4f147c13d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.529821 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07afb0f1b47e46ae9625eac4f147c13d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:20.531820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:20.590714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07afb0f1b47e46ae9625eac4f147c13d', 'request_id': 'req-7028fed6-e6be-44f9-aaff-bee67d1f5f28'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.597190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07afb0f1b47e46ae9625eac4f147c13d', 'request_id': 'req-7028fed6-e6be-44f9-aaff-bee67d1f5f28'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.603855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07afb0f1b47e46ae9625eac4f147c13d', 'request_id': 'req-7028fed6-e6be-44f9-aaff-bee67d1f5f28'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.619699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] The token cache is being invalidate because project 07afb0f1b47e46ae9625eac4f147c13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:20.620393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.621054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7028fed6-e6be-44f9-aaff-bee67d1f5f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:20.626470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1240/2486] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:20 2022] DELETE /identity/v3/projects/07afb0f1b47e46ae9625eac4f147c13d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:20.631770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.632257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.632366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.656498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:20.657898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:20.745185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1247/2487] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:44:20.747561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.747886 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.748282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.774296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:20.775532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:20.785917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1241/2488] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:20.790446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:20.806245 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504e562-11cc-40f9-ae53-0cfa0daeb358 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:20.813763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:20.814617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.814990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.815906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.816749 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:20.817185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:20.821078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:20.838969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1248/2489] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:20.843432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:20.853068 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435d6a-2300-4d34-8978-a8728920d054 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:20.863220 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:20.863944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.864424 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.864918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.865712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:20.868631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:20.904217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1242/2490] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:20 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:20.908417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:20.918991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3782-8996-4362-a786-d5ab685db7af admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:20.986572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:20.987259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:20.987539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:20.987869 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:20.988676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:20.991818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:21.025427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1249/2491] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:20 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:21.029489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:21.041228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e009a-2c53-48da-9aa9-fdb8bbf3db04 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:21.046233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:21.046860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:21.047139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:21.047411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:21.048240 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:21.049779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:21.061538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1243/2492] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:21.065572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:21.076336 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368fd62-be3c-4bf7-afd3-a98393c0be50 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:21.082208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd864dd-1e9d-43d6-b3f7-32dfe15e44d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:21.082208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd864dd-1e9d-43d6-b3f7-32dfe15e44d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:21.082431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd864dd-1e9d-43d6-b3f7-32dfe15e44d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:21.083384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd864dd-1e9d-43d6-b3f7-32dfe15e44d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:21.086427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd864dd-1e9d-43d6-b3f7-32dfe15e44d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:21.086427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd864dd-1e9d-43d6-b3f7-32dfe15e44d3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:21.092017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:21.102818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1250/2493] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:21.103414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:21.114090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870eb5c-6563-4963-9516-15bb6b516bf1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:21.118662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:21.119327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:21.119598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:21.119895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] PATH_INFO: `/v3/projects/2f984cdd4a16415eb88d62b9a8b871df/users/b03a04b1556e4d9e9e87357375f6db80/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:21.135173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f984cdd4a16415eb88d62b9a8b871df, user_id=b03a04b1556e4d9e9e87357375f6db80, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:21.137974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:21.147616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:21.152812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1244/2494] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:21 2022] PUT /identity/v3/projects/2f984cdd4a16415eb88d62b9a8b871df/users/b03a04b1556e4d9e9e87357375f6db80/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:21.163403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1146df-d04f-4438-977a-7860d4c9e489 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:21.163807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1146df-d04f-4438-977a-7860d4c9e489 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:21.164142 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1146df-d04f-4438-977a-7860d4c9e489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:21.195248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b1146df-d04f-4438-977a-7860d4c9e489 None None] MFA Rules not processed for user `b03a04b1556e4d9e9e87357375f6db80`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:21.196459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b1146df-d04f-4438-977a-7860d4c9e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:21.295003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1251/2495] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:21.303441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:21.305249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:21.363857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9438ab-4d4f-4325-bef2-783decdd9573 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:21.367768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:21.368395 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:21.368663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:21.368972 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:21.378119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:21.379577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:21.389776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1245/2496] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:21.881247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:21.882173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:21.882447 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:21.905473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] MFA Rules not processed for user `f5d3517f51b74230964f1e2b29371e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:21.906953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:21.973108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1252/2497] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:22.359381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:22.362952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:22.406632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:22.409728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:22.451996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566549a4-9ac6-4318-a014-d92525bd1a6d service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:22.456592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:22.457339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:22.457686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:22.458022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] PATH_INFO: `/v3/users/120c06d6dee6425ea5366b363d5a5f7e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:22.459287 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=120c06d6dee6425ea5366b363d5a5f7e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:22.461355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:22.485347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ee0a9-6817-4889-8d57-0b4a539921df None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:22.491326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:22.492260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:22.495845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:22.495845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:22.495845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:22.507833 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:22.509407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:22.519922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1253/2498] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:44:22.530284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120c06d6dee6425ea5366b363d5a5f7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:22.539930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120c06d6dee6425ea5366b363d5a5f7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:22.540647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120c06d6dee6425ea5366b363d5a5f7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:22.555115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120c06d6dee6425ea5366b363d5a5f7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:22.568324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120c06d6dee6425ea5366b363d5a5f7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:22.576122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1246/2499] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:22 2022] DELETE /identity/v3/users/120c06d6dee6425ea5366b363d5a5f7e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:22.792211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:22.810793 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2600a9-689a-4237-98ad-d7e12cb09ec0 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:22.847706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:22.883759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:22.885374 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:22.885843 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:22.886296 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] PATH_INFO: `/v3/projects/2269cace554c401a8b6a6303ff4e4da8` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:22.887935 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2269cace554c401a8b6a6303ff4e4da8)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:22.896728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:22.990444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:22.991902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.007949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2269cace554c401a8b6a6303ff4e4da8', 'request_id': 'req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.014930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2269cace554c401a8b6a6303ff4e4da8', 'request_id': 'req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.019033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cc8e1-32e0-41af-83ca-68247a3ec4f8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.023385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2269cace554c401a8b6a6303ff4e4da8', 'request_id': 'req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.043936 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] The token cache is being invalidate because project 2269cace554c401a8b6a6303ff4e4da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:23.044787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.046301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-435ef0c9-aaa0-453e-ac34-2448a6d7ce25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.048652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1254/2500] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:22 2022] DELETE /identity/v3/projects/2269cace554c401a8b6a6303ff4e4da8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:23.054249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe839-fd94-422b-8c72-a01b7b212294 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.054760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe839-fd94-422b-8c72-a01b7b212294 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.055155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe839-fd94-422b-8c72-a01b7b212294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.060727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:23.087055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-fd6fe839-fd94-422b-8c72-a01b7b212294 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:23.088391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fd6fe839-fd94-422b-8c72-a01b7b212294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.115681 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.116515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.116823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.117097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] PATH_INFO: `/v3/users/7f7f0d28f8874bdfa9e2f726c9ae62ec` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.118572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7f0d28f8874bdfa9e2f726c9ae62ec)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.120852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.170105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1255/2501] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:23.179639 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.180253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.180418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7f0d28f8874bdfa9e2f726c9ae62ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.180889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.192353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7f0d28f8874bdfa9e2f726c9ae62ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.194707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7f0d28f8874bdfa9e2f726c9ae62ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.208787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:23.209991 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.214283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7f0d28f8874bdfa9e2f726c9ae62ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.221823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1256/2502] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:23.224598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.229682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7db366e-b349-48c6-95f8-e39268dfede6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7f0d28f8874bdfa9e2f726c9ae62ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:23.236756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655b2ca-2654-47d7-8053-03c57cff1017 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.237854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1247/2503] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:22 2022] DELETE /identity/v3/users/7f7f0d28f8874bdfa9e2f726c9ae62ec => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:23.242618 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.243154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.243368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.271452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:23.272693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.308642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.309924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.309924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.310103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.313146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:23.313694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.315914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.319025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:23.338201 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1257/2504] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:44:23.342961 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.344565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1248/2505] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:23.346688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.358364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.360129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.361268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f47e-5913-477f-9c04-edcd92b06748 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.367692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.368317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.368669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.368950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.369780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.371886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.409269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1258/2506] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:23.413788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.437627 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:23.456377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bacdd2-01b4-457c-bf45-824cdd05304c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.499848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a8a28-7f80-4ee0-8454-be34c2a66a6b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.506649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.507278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.507555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.507824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.509016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.511685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.520530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.521280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.521559 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.521834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.535216 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.536071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1259/2507] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/users => generated 303 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:23.538034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.540578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.558494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1249/2508] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:23.567812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd7c2-dc9b-4b87-82fa-07d89149a975 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.573905 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.574617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.574958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.575401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.576536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.578559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.595222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1260/2509] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:23.596461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.608022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d298e-22d5-44d8-8341-6a75b8b49d85 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.614732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf633de-5f1a-4268-9996-f34ba0373d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.615682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf633de-5f1a-4268-9996-f34ba0373d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.616077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf633de-5f1a-4268-9996-f34ba0373d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.616752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf633de-5f1a-4268-9996-f34ba0373d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.617855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf633de-5f1a-4268-9996-f34ba0373d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.620045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf633de-5f1a-4268-9996-f34ba0373d58 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.650381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1250/2510] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:23.652133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.662972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f54add-41b4-4f46-95d8-7155e6882156 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.668724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.669857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.670278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.671231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] PATH_INFO: `/v3/projects/1c3e4796f65444a1ab27c232782cba1c/users/1c2f0da9d0ff4ac093b523db63846c2d/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.696016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3e4796f65444a1ab27c232782cba1c, user_id=1c2f0da9d0ff4ac093b523db63846c2d, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.699969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.718455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1261/2511] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:23 2022] PUT /identity/v3/projects/1c3e4796f65444a1ab27c232782cba1c/users/1c2f0da9d0ff4ac093b523db63846c2d/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:23.729520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.729802 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.730079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.762909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] MFA Rules not processed for user `1c2f0da9d0ff4ac093b523db63846c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:23.764133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.832532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1251/2512] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:23.837541 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.903092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:23.905122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:23.912976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf335-ce32-472d-959d-4a0f273d2ec1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.922291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.923003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.923439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.923715 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.925134 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.933566 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035d4e-9c8f-4427-8da7-613e32f4f1cc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:23.934276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.939488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:23.940314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:23.940655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:23.941030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] PATH_INFO: `/v3/projects/52f5c75ba0e144728a118db8ecf63cf4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:23.942700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f5c75ba0e144728a118db8ecf63cf4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:23.944754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:23.973934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1262/2513] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:23.978424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.008252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:24.051591 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f5c75ba0e144728a118db8ecf63cf4', 'request_id': 'req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.063095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f5c75ba0e144728a118db8ecf63cf4', 'request_id': 'req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.084883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f5c75ba0e144728a118db8ecf63cf4', 'request_id': 'req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.085461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040ce5-0a3e-4dd7-9784-d6ea120c94d4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.102369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] The token cache is being invalidate because project 52f5c75ba0e144728a118db8ecf63cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:24.102732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.103400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.105231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1252/2514] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:23 2022] DELETE /identity/v3/projects/52f5c75ba0e144728a118db8ecf63cf4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:24.109532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.110975 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.116149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:24.161080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.161804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.162079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.162349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.163014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.165409 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.198546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1263/2515] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:23 2022] POST /identity/v3/users => generated 309 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:24.199496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.201480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.205428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd969d1-e9bd-4ddc-ae29-49a4b9a39f11 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.212222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.213527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.213857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.214135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] PATH_INFO: `/v3/users/0a454768f19c4b2aaf4f98b9048adfc0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.215388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a454768f19c4b2aaf4f98b9048adfc0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.217387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.226358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d9cbe-fe2b-4ffc-9b2d-31198f4d77b4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.234844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.235594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.236104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.236213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.237056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.238765 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.250970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1264/2516] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:24.256089 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.266700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b532a1-ac7a-47d4-ae71-1f23a06febb1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.272699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.273475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.273750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.274021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] PATH_INFO: `/v3/projects/1f5f94232c0a4382822b8fa1b00eee72/users/fd99da05e41b4d73ae17316ca8f8263d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.278326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a454768f19c4b2aaf4f98b9048adfc0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.286889 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a454768f19c4b2aaf4f98b9048adfc0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.287571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a454768f19c4b2aaf4f98b9048adfc0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.292701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5f94232c0a4382822b8fa1b00eee72, user_id=fd99da05e41b4d73ae17316ca8f8263d, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.295483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.304554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a454768f19c4b2aaf4f98b9048adfc0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.311011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1265/2517] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:24 2022] PUT /identity/v3/projects/1f5f94232c0a4382822b8fa1b00eee72/users/fd99da05e41b4d73ae17316ca8f8263d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:24.314834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a454768f19c4b2aaf4f98b9048adfc0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.315093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.316703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.322125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1253/2518] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:24 2022] DELETE /identity/v3/users/0a454768f19c4b2aaf4f98b9048adfc0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:24.395726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a980d-cb9f-475c-9661-59632e473a91 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.420233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:24.467960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c089-23ba-43ce-9480-937427bce880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.468596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c089-23ba-43ce-9480-937427bce880 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.469642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c089-23ba-43ce-9480-937427bce880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.469642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c089-23ba-43ce-9480-937427bce880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.469923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c089-23ba-43ce-9480-937427bce880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.471526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c089-23ba-43ce-9480-937427bce880 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.483614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1266/2519] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:24.487699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.505476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce0bde-b2c9-40ff-abef-431466981165 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.511524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.512820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.512820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.512820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] PATH_INFO: `/v3/projects/1f5f94232c0a4382822b8fa1b00eee72/users/fd99da05e41b4d73ae17316ca8f8263d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.514909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5f94232c0a4382822b8fa1b00eee72, user_id=fd99da05e41b4d73ae17316ca8f8263d, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.517685 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.529890 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1254/2520] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:24 2022] PUT /identity/v3/projects/1f5f94232c0a4382822b8fa1b00eee72/users/fd99da05e41b4d73ae17316ca8f8263d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:24.541667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.541951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.542223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.571269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] MFA Rules not processed for user `fd99da05e41b4d73ae17316ca8f8263d`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:24.572490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.618278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.619987 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.673427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1267/2521] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:24.684220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.685647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.707200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0969f-e801-48d3-951a-f784ad3472c0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.713599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.714366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.714640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.714906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] PATH_INFO: `/v3/projects/1de7da55d0904421bd38884790dc90b5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.716593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de7da55d0904421bd38884790dc90b5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.718544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.732528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:24.760702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb647372-7fdb-4df6-9293-f87d20debc3c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.766237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.769635 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.770746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.771097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.786669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.786825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de7da55d0904421bd38884790dc90b5', 'request_id': 'req-02712b9a-2495-43ed-862d-860acd0c3c96'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.788109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.793430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de7da55d0904421bd38884790dc90b5', 'request_id': 'req-02712b9a-2495-43ed-862d-860acd0c3c96'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.798904 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1268/2522] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:44:24.799571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:24.800994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.803364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de7da55d0904421bd38884790dc90b5', 'request_id': 'req-02712b9a-2495-43ed-862d-860acd0c3c96'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.817250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd3b04-cce8-4564-a67e-4854f6a9f4ca service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:24.820608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] The token cache is being invalidate because project 1de7da55d0904421bd38884790dc90b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:24.821115 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.822102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-02712b9a-2495-43ed-862d-860acd0c3c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.825273 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1255/2523] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:24 2022] DELETE /identity/v3/projects/1de7da55d0904421bd38884790dc90b5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:24.834917 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a51ec6-77fd-46ac-813a-0f80a7d61b08 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.836051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a51ec6-77fd-46ac-813a-0f80a7d61b08 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.836505 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a51ec6-77fd-46ac-813a-0f80a7d61b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.867487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b6a51ec6-77fd-46ac-813a-0f80a7d61b08 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:24.869228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6a51ec6-77fd-46ac-813a-0f80a7d61b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:24.882493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:24.901075 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:24.901637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.901939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.902182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] PATH_INFO: `/v3/users/b03a04b1556e4d9e9e87357375f6db80` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.903488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03a04b1556e4d9e9e87357375f6db80)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:24.906558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:24.959260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1256/2524] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:24.967755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:24.968454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:24.968454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:24.983281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03a04b1556e4d9e9e87357375f6db80'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.993478 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03a04b1556e4d9e9e87357375f6db80'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.994254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03a04b1556e4d9e9e87357375f6db80'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:24.996552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:24.997909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.003490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03a04b1556e4d9e9e87357375f6db80'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.004255 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03a04b1556e4d9e9e87357375f6db80'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.012867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1257/2525] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:44:25.012867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.012867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.023731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32b6c6-4125-44b9-a69d-8867909c3c62 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.029950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1269/2526] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:24 2022] DELETE /identity/v3/users/b03a04b1556e4d9e9e87357375f6db80 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:25.030933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.032483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.033610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.033929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.034285 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] PATH_INFO: `/v3/users/1c2f0da9d0ff4ac093b523db63846c2d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.035789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2f0da9d0ff4ac093b523db63846c2d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.037977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.122912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2f0da9d0ff4ac093b523db63846c2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.123147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ec5f1-6279-42f7-9d62-a2f37c7a10ae admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.129138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.129863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.130144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.130444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.131143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:25.131562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.133371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a0d6f-e71e-40b3-8acc-65228e61fd76 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.136531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2f0da9d0ff4ac093b523db63846c2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.141416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2f0da9d0ff4ac093b523db63846c2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.148455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1270/2527] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:25.149891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2476c-b913-47f7-a44b-41498753588d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.150176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2476c-b913-47f7-a44b-41498753588d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.150441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2476c-b913-47f7-a44b-41498753588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.161921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2f0da9d0ff4ac093b523db63846c2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.169533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2f0da9d0ff4ac093b523db63846c2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:25.173750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cbe2476c-b913-47f7-a44b-41498753588d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:25.175007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cbe2476c-b913-47f7-a44b-41498753588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.178724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1258/2528] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:25 2022] DELETE /identity/v3/users/1c2f0da9d0ff4ac093b523db63846c2d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:25.179320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.180801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.187109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1271/2529] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:25.187109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.187109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.187109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.190279 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:25.208107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:25.209244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.271107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6c104-77b5-4595-a2e7-966a5e1432e6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.276616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.277176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:25.277377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.277608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.278089 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.278943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.281498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.297906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1272/2530] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:25.299004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.300519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.317266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1259/2531] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:25 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:25.318516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.322030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.377886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745ec82-2bc2-4072-be89-f85a08f360fe None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.382709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.383432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.383775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.384116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.390868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98117a83-2273-4dd0-82ec-bb2611907c49 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.395862 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.397326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.397605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.397884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.454959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.456398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.469065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1273/2532] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:25.470383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.482073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca54e65-1a1e-4b5a-8067-0ff6115b0c2c service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.493507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.494511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.495395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.495713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.496084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.496558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.497104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.499471 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.513437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1260/2533] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:25.528059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1274/2534] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:25 2022] POST /identity/v3/users => generated 284 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:25.529551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.540530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faee205-98a7-46a6-87a0-30f0a295bec2 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.545712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.546341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.546613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.546878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.547804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.550151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.564337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1261/2535] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:25.565725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.579778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157f714-48ce-40c7-97bf-e85742f2f425 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.587314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f712c-55e8-41ca-8721-4174bee1b664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.587959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f712c-55e8-41ca-8721-4174bee1b664 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.588470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f712c-55e8-41ca-8721-4174bee1b664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.589129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f712c-55e8-41ca-8721-4174bee1b664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.589546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f712c-55e8-41ca-8721-4174bee1b664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.591271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f712c-55e8-41ca-8721-4174bee1b664 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.605656 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1275/2536] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:25.611011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.637157 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4c4a4-68e2-41b6-ba55-10340e0af1df admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.645872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.647222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.647513 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.647796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] PATH_INFO: `/v3/projects/c2eaf3a31bc3456f8302f4600d06cedb/users/fda34326dbf940c6906817160e78f4a0/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.671029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2eaf3a31bc3456f8302f4600d06cedb, user_id=fda34326dbf940c6906817160e78f4a0, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.674336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:25.707028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1262/2537] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:25 2022] PUT /identity/v3/projects/c2eaf3a31bc3456f8302f4600d06cedb/users/fda34326dbf940c6906817160e78f4a0/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:25.709520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.710721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.711086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.752650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] MFA Rules not processed for user `fda34326dbf940c6906817160e78f4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:25.754423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:25.867260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1276/2538] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:25.868825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:25.878678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:25.971705 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7313409-f56a-4a30-bc89-18c5d798313f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:25.979074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:25.980113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:25.980113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:25.980113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:25.980742 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:25.982783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.018645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1263/2539] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:25 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:26.024617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.038442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:26.043405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2418fd8-7617-4c52-bcfc-a29a8beccff0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.047565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.049647 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.139490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.140333 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.140611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.140911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.142055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.143526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.148453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fd5e3-e6a5-4d6a-a01d-fa213f749ab2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.153246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.153899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.154179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.154448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] PATH_INFO: `/v3/projects/2f984cdd4a16415eb88d62b9a8b871df` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.155756 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f984cdd4a16415eb88d62b9a8b871df)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.157708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.170491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1277/2540] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:26 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:26.171723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.174782 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.189997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b0ce-db1b-40f3-a794-c88efa9d89ae admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.196695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.197367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.197644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.199176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] PATH_INFO: `/v3/projects/1c3e4796f65444a1ab27c232782cba1c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.200558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3e4796f65444a1ab27c232782cba1c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.202591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.212611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f984cdd4a16415eb88d62b9a8b871df', 'request_id': 'req-a8e3534b-8a70-4e69-8f56-8e4096168b24'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.225219 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f984cdd4a16415eb88d62b9a8b871df', 'request_id': 'req-a8e3534b-8a70-4e69-8f56-8e4096168b24'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.231824 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f984cdd4a16415eb88d62b9a8b871df', 'request_id': 'req-a8e3534b-8a70-4e69-8f56-8e4096168b24'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.247620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] The token cache is being invalidate because project 2f984cdd4a16415eb88d62b9a8b871df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:26.247906 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.248541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.253563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1264/2541] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:26 2022] DELETE /identity/v3/projects/2f984cdd4a16415eb88d62b9a8b871df => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:26.253563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.253563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.266103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3e4796f65444a1ab27c232782cba1c', 'request_id': 'req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.272997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3e4796f65444a1ab27c232782cba1c', 'request_id': 'req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.276047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:26.279563 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3e4796f65444a1ab27c232782cba1c', 'request_id': 'req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.298165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] The token cache is being invalidate because project 1c3e4796f65444a1ab27c232782cba1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:26.298630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.299745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f93d7a18-d1a2-4e21-b6fa-89490f1c8385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.302056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1278/2542] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:26 2022] DELETE /identity/v3/projects/1c3e4796f65444a1ab27c232782cba1c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:26.303615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.304004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.304964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.329024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:26.330209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.330400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3534b-8a70-4e69-8f56-8e4096168b24 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.399508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8087-3557-4d21-9e5c-b01999c12e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.400055 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8087-3557-4d21-9e5c-b01999c12e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.400329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8087-3557-4d21-9e5c-b01999c12e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.400596 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8087-3557-4d21-9e5c-b01999c12e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.401323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b8087-3557-4d21-9e5c-b01999c12e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.402252 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1279/2543] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:26.402927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b8087-3557-4d21-9e5c-b01999c12e32 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.403096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.404350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.416005 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dac4ce-7500-4616-abde-781440d01528 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.420896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1265/2544] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:26.420896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.420896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.420896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.424974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.425628 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.425906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.426173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] PATH_INFO: `/v3/users/fd99da05e41b4d73ae17316ca8f8263d` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.427448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd99da05e41b4d73ae17316ca8f8263d)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.429476 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.441598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:26.442872 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.445930 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:26.451452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1266/2545] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:26.455290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.455290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.465690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc2751-b57b-4207-a464-c12aac12348f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.471059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.471706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.471988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.472238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] PATH_INFO: `/v3/projects/f8d66169053342ed9a1f26c8395676c8/users/e9252a9741c748048d0be85afdc03247/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.488163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d66169053342ed9a1f26c8395676c8, user_id=e9252a9741c748048d0be85afdc03247, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.491043 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.496681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd99da05e41b4d73ae17316ca8f8263d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.508753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1267/2546] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:26 2022] PUT /identity/v3/projects/f8d66169053342ed9a1f26c8395676c8/users/e9252a9741c748048d0be85afdc03247/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:26.508753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.511652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd99da05e41b4d73ae17316ca8f8263d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.512338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd99da05e41b4d73ae17316ca8f8263d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.527960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd99da05e41b4d73ae17316ca8f8263d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.528733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd99da05e41b4d73ae17316ca8f8263d'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.550484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1280/2547] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:26 2022] DELETE /identity/v3/users/fd99da05e41b4d73ae17316ca8f8263d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:26.551836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.567419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.572153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795ff38-34b7-4642-a7ff-b9b327ed6ea5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.615891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:26.640838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.641474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.642109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.642109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.642854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:26.643057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.644738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.658432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1268/2548] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:26.663207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.665228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.668144 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759fca0-de7a-4386-a26f-3cd82f8e24c4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.673002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.673630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.673906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.674379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.675080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.676682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.680345 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37e647-eb15-4b2e-bffe-de3dbc987152 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.685950 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.686899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.687328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.687760 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.691022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1281/2549] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:26.691179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.691529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.693701 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.706048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d26fa-f00b-4d22-9a44-de7e241a2d4a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.716526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9006b-ea8d-48bd-bafe-1b3db6f429cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.717455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9006b-ea8d-48bd-bafe-1b3db6f429cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.718278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9006b-ea8d-48bd-bafe-1b3db6f429cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.718278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9006b-ea8d-48bd-bafe-1b3db6f429cb admin admin] PATH_INFO: `/v3/projects/f8d66169053342ed9a1f26c8395676c8/users/e9252a9741c748048d0be85afdc03247/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.721234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9006b-ea8d-48bd-bafe-1b3db6f429cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d66169053342ed9a1f26c8395676c8, user_id=e9252a9741c748048d0be85afdc03247, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.724669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9006b-ea8d-48bd-bafe-1b3db6f429cb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.729404 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1269/2550] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:26 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:26.730401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.746237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1282/2551] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:26 2022] PUT /identity/v3/projects/f8d66169053342ed9a1f26c8395676c8/users/e9252a9741c748048d0be85afdc03247/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:26.748819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.749182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.749517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.782007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] MFA Rules not processed for user `e9252a9741c748048d0be85afdc03247`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:26.783443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.800771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4ee01-4661-4d67-8ed0-e5e7931aa7d1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.857733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:26.863864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1283/2552] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:26.864847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.866788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.871761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.872476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.872853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.873248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.874001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.876066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.885725 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca31c96-c4b3-466c-b75f-2ecae815a559 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:26.891467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:26.892198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:26.892531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:26.892917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] PATH_INFO: `/v3/projects/1f5f94232c0a4382822b8fa1b00eee72` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:26.894164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5f94232c0a4382822b8fa1b00eee72)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:26.896227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:26.901665 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1270/2553] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:26 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:26.902561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.904121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:26.949146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5f94232c0a4382822b8fa1b00eee72', 'request_id': 'req-2122c55c-a6f0-4dd5-9dce-5b30c475e721'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.956502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5f94232c0a4382822b8fa1b00eee72', 'request_id': 'req-2122c55c-a6f0-4dd5-9dce-5b30c475e721'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.963898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5f94232c0a4382822b8fa1b00eee72', 'request_id': 'req-2122c55c-a6f0-4dd5-9dce-5b30c475e721'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.979808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] The token cache is being invalidate because project 1f5f94232c0a4382822b8fa1b00eee72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:26.980105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.980891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:26.982513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1284/2554] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:26 2022] DELETE /identity/v3/projects/1f5f94232c0a4382822b8fa1b00eee72 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:26.986229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:26.986229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.056642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122c55c-a6f0-4dd5-9dce-5b30c475e721 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.062694 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21549bda-abcf-4500-a2c3-7dd4072800e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.063388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21549bda-abcf-4500-a2c3-7dd4072800e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.063633 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21549bda-abcf-4500-a2c3-7dd4072800e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.063926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21549bda-abcf-4500-a2c3-7dd4072800e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.064629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21549bda-abcf-4500-a2c3-7dd4072800e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.066279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21549bda-abcf-4500-a2c3-7dd4072800e4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.077479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1285/2555] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:27.081821 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.082090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.082357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.093107 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:27.097446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717073-19f1-4470-8679-8c92f6617863 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.106791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:27.108155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.116888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1286/2556] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:27.121009 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.132132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c542a4-5967-4b1f-80e6-2da8d3080a5a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.139464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906c285-9ef2-423c-958f-a471ec6839d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.139464 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906c285-9ef2-423c-958f-a471ec6839d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.139836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906c285-9ef2-423c-958f-a471ec6839d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.140589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906c285-9ef2-423c-958f-a471ec6839d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.141386 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906c285-9ef2-423c-958f-a471ec6839d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.143784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906c285-9ef2-423c-958f-a471ec6839d6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.150563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.151561 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.152499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.152942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.155041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1287/2557] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:27.155884 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.158265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.158550 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.158824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.180892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:27.182138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.191017 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1288/2558] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:27.191844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.203600 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ed059-9dc0-4b34-ae30-05e363524a73 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.208730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867d25-6286-48a5-8677-25922db46792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.209445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867d25-6286-48a5-8677-25922db46792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.209714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867d25-6286-48a5-8677-25922db46792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.209981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867d25-6286-48a5-8677-25922db46792 admin admin] PATH_INFO: `/v3/projects/a5fed4f0efcc4c7184029437668e4286/users/c9c5d3fa5a4b45d79ea7f6fd5dde2997/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.230699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.232457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.237888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867d25-6286-48a5-8677-25922db46792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fed4f0efcc4c7184029437668e4286, user_id=c9c5d3fa5a4b45d79ea7f6fd5dde2997, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.240718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08867d25-6286-48a5-8677-25922db46792 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.244852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1271/2559] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:27.249730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.261856 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50242a-72c3-45af-8ee2-4941f4afe4f1 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.262343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1289/2560] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:27 2022] PUT /identity/v3/projects/a5fed4f0efcc4c7184029437668e4286/users/c9c5d3fa5a4b45d79ea7f6fd5dde2997/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:27.272981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.273279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.273581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.314038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] MFA Rules not processed for user `c9c5d3fa5a4b45d79ea7f6fd5dde2997`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:27.315258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.334202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:27.341222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.341880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.342223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.342439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.343128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:27.343427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.345082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.357432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1272/2561] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:27.358347 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.360077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.396406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1290/2562] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:44:27.397539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.409527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03526662-5d25-4b7d-8f21-146657b416ad None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.416195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.417289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.417583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.418433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.418576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.420498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.423984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:27.428533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964ddc-8640-472c-96f4-392db05defdb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.434471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.435127 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.435403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.435672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.437573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.459119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1291/2563] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:27.459119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.459119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.502848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.504479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.513866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1273/2564] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:27.516527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.521498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615da37e-be78-4d53-98b4-1e953bc25de4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.526984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5cd94-b402-4d0e-8ff3-2a71abfa7b51 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.527307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.528709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.529066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.529291 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.588238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.589147 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.589423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.589689 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.590494 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.592979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.600192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.601659 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.611467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1292/2565] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:27.617382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1274/2566] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:27.622143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.642524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e5520-8872-4806-8db9-a959c8114b22 service glance] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.647350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.648475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.648475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.648702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.649974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.651054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.664827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1293/2567] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:27.666156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.687995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422061-9559-4192-b18a-85d3dec8e524 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.693058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78549b62-c33a-42ba-990b-52d604659d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.693694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78549b62-c33a-42ba-990b-52d604659d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.693969 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78549b62-c33a-42ba-990b-52d604659d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.694234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78549b62-c33a-42ba-990b-52d604659d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.694935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78549b62-c33a-42ba-990b-52d604659d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.696542 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78549b62-c33a-42ba-990b-52d604659d0a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.716086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1275/2568] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:27.720964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.740949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761350-8feb-4140-98ab-2ef33ed3d75f admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:27.755391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:27.756087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.756368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.756642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] PATH_INFO: `/v3/projects/fb653b1d1cf04fe8b27b2fb6004ee995/users/ec03c7db866645a2b6de6e9114cd7d99/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.775047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb653b1d1cf04fe8b27b2fb6004ee995, user_id=ec03c7db866645a2b6de6e9114cd7d99, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:27.778505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:27.799325 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1294/2569] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:27 2022] PUT /identity/v3/projects/fb653b1d1cf04fe8b27b2fb6004ee995/users/ec03c7db866645a2b6de6e9114cd7d99/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:27.809632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:27.810118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:27.810392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:27.844399 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] MFA Rules not processed for user `ec03c7db866645a2b6de6e9114cd7d99`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:27.845866 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:27.940339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1276/2570] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:27.940655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:27.962889 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:28.013994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1efbb9-b504-4788-aef4-785bda9128c4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.018618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.019234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.019536 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.019810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.020498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.022442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.050627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1295/2571] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:27 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:28.055201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.066482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d1617-cab1-4bdd-8468-0da1ff28414d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.081222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:28.143912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.144546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.144847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.145113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.145774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.147684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.172000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1277/2572] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:28 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:28.173223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.184734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5fbe9-a3b2-4132-817e-42ca45be5885 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.189443 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.190050 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.190323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.190618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.191334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.192971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.203589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1296/2573] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:28.207897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.219012 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe58f5a-8e25-4c31-8c38-bd39f151a08f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.224200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.224506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.224850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.225120 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] PATH_INFO: `/v3/projects/578630bf0d4f466b881c279a3b9a80ed/users/69921b470f7b4315b241802618903a5a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.240093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578630bf0d4f466b881c279a3b9a80ed, user_id=69921b470f7b4315b241802618903a5a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.242709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.256158 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1278/2574] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:28 2022] PUT /identity/v3/projects/578630bf0d4f466b881c279a3b9a80ed/users/69921b470f7b4315b241802618903a5a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:28.260316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.270836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe6153-9abc-4fa9-981a-d3dea0fe4504 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.330589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab5bb2-4a06-46b6-b346-62acdac6b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.331212 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab5bb2-4a06-46b6-b346-62acdac6b26c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.331487 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab5bb2-4a06-46b6-b346-62acdac6b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.331755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab5bb2-4a06-46b6-b346-62acdac6b26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.332507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab5bb2-4a06-46b6-b346-62acdac6b26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.334136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab5bb2-4a06-46b6-b346-62acdac6b26c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.344021 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1297/2575] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:28.348447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.359100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4e1fe-00e5-4ff7-9d38-022afc262096 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.364114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.364748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.365070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.365334 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] PATH_INFO: `/v3/projects/578630bf0d4f466b881c279a3b9a80ed/users/69921b470f7b4315b241802618903a5a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.367463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] RBAC: Authorizing `identity:create_grant(project_id=578630bf0d4f466b881c279a3b9a80ed, user_id=69921b470f7b4315b241802618903a5a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.370122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.381772 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1279/2576] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:28 2022] PUT /identity/v3/projects/578630bf0d4f466b881c279a3b9a80ed/users/69921b470f7b4315b241802618903a5a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:28.393792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.394056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.394354 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.426379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] MFA Rules not processed for user `69921b470f7b4315b241802618903a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:28.427813 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:28.509483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1298/2577] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:28.520912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.522654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:28.583173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf091b0-b507-41e9-9ccf-7e02b9dbd68a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.587588 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.588312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.588585 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.588922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.598787 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.600391 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.612431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1280/2578] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:28.665116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:28.675343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a96f2-9c26-4db8-9d3a-4514bad9aa63 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:28.679363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0050ef7-34b0-464a-ab49-ae5325b7db3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:28.680117 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0050ef7-34b0-464a-ab49-ae5325b7db3a service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:28.680393 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0050ef7-34b0-464a-ab49-ae5325b7db3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:28.680687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0050ef7-34b0-464a-ab49-ae5325b7db3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:28.690098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:28.745395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0050ef7-34b0-464a-ab49-ae5325b7db3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:28.746871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0050ef7-34b0-464a-ab49-ae5325b7db3a service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:28.756565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1299/2579] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:31.193532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:31.195117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:31.270949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30afaa5-beb2-47b6-9e68-ec9397993380 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:31.276190 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:31.277030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:31.277290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:31.277583 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] PATH_INFO: `/v3/users/fda34326dbf940c6906817160e78f4a0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:31.278822 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda34326dbf940c6906817160e78f4a0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:31.280953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:31.288447 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:31.347916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda34326dbf940c6906817160e78f4a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.355658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda34326dbf940c6906817160e78f4a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.356615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda34326dbf940c6906817160e78f4a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.368844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda34326dbf940c6906817160e78f4a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.376258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda34326dbf940c6906817160e78f4a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.393968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1281/2580] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:31 2022] DELETE /identity/v3/users/fda34326dbf940c6906817160e78f4a0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:31.398666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:31.399123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:31.399221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:31.418402 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:31.419601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:31.497360 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1300/2581] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:31.507857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:31.509535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:31.567968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13b440-768a-45b6-90d4-ad4e2407e80a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:31.571790 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:31.572400 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:31.572653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:31.572973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:31.582007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:31.583434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:31.593083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1282/2582] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:31.872136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:31.873687 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:31.887233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11914b-ca67-4cf6-8502-7a571714bed0 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:31.892746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:31.893498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:31.893840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:31.894168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] PATH_INFO: `/v3/projects/c2eaf3a31bc3456f8302f4600d06cedb` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:31.896125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2eaf3a31bc3456f8302f4600d06cedb)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:31.898545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:31.949103 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2eaf3a31bc3456f8302f4600d06cedb', 'request_id': 'req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.955457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2eaf3a31bc3456f8302f4600d06cedb', 'request_id': 'req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.960953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2eaf3a31bc3456f8302f4600d06cedb', 'request_id': 'req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.974067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] The token cache is being invalidate because project c2eaf3a31bc3456f8302f4600d06cedb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:31.974352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.974980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:31.976816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1301/2583] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:31 2022] DELETE /identity/v3/projects/c2eaf3a31bc3456f8302f4600d06cedb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:31.981496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:31.992328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5930b-0942-4e36-9dd1-d26086dd79c0 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.054042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.054699 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.054976 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.055248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] PATH_INFO: `/v3/users/e9252a9741c748048d0be85afdc03247` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.056427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9252a9741c748048d0be85afdc03247)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.058545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.094966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:32.110033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9252a9741c748048d0be85afdc03247'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.117859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9252a9741c748048d0be85afdc03247'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.118739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9252a9741c748048d0be85afdc03247'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.131840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9252a9741c748048d0be85afdc03247'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.138901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9252a9741c748048d0be85afdc03247'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.145578 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1283/2584] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:31 2022] DELETE /identity/v3/users/e9252a9741c748048d0be85afdc03247 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:32.325560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.327218 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:32.340318 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3c70-6fd3-4d39-b2cf-8823ea0549a3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.377490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:32.412159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.412745 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.413004 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.413308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] PATH_INFO: `/v3/users/ec03c7db866645a2b6de6e9114cd7d99` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.414490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec03c7db866645a2b6de6e9114cd7d99)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.416741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.463983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.466274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:32.484060 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec03c7db866645a2b6de6e9114cd7d99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.489734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808475-7744-4b9b-aa18-3922d5215394 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.496210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec03c7db866645a2b6de6e9114cd7d99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.497578 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec03c7db866645a2b6de6e9114cd7d99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.502877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.503648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.504007 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.504299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] PATH_INFO: `/v3/projects/f8d66169053342ed9a1f26c8395676c8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.505572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d66169053342ed9a1f26c8395676c8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.508343 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.511491 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec03c7db866645a2b6de6e9114cd7d99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.512750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0cf8b14a-742d-469f-9245-cda3559fea3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec03c7db866645a2b6de6e9114cd7d99'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.552937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1302/2585] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:32 2022] DELETE /identity/v3/users/ec03c7db866645a2b6de6e9114cd7d99 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:32.555310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d538b-914a-4093-98df-e088e71b2582 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.555606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d538b-914a-4093-98df-e088e71b2582 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.555892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d538b-914a-4093-98df-e088e71b2582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.570621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d66169053342ed9a1f26c8395676c8', 'request_id': 'req-ad81856b-ece4-4bc4-84b5-2827d247c89a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.578835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d66169053342ed9a1f26c8395676c8', 'request_id': 'req-ad81856b-ece4-4bc4-84b5-2827d247c89a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.581240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-328d538b-914a-4093-98df-e088e71b2582 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:32.582537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-328d538b-914a-4093-98df-e088e71b2582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:32.585614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d66169053342ed9a1f26c8395676c8', 'request_id': 'req-ad81856b-ece4-4bc4-84b5-2827d247c89a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.617729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] The token cache is being invalidate because project f8d66169053342ed9a1f26c8395676c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:32.618083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.618913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad81856b-ece4-4bc4-84b5-2827d247c89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:32.625230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1284/2586] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:32 2022] DELETE /identity/v3/projects/f8d66169053342ed9a1f26c8395676c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:32.628651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1804354-2586-4e70-8024-f9c42b7de089 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.628936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1804354-2586-4e70-8024-f9c42b7de089 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.629223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1804354-2586-4e70-8024-f9c42b7de089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.656146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e1804354-2586-4e70-8024-f9c42b7de089 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:32.657396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1804354-2586-4e70-8024-f9c42b7de089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:32.666812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1303/2587] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:44:32.677822 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d538b-914a-4093-98df-e088e71b2582 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.679256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-328d538b-914a-4093-98df-e088e71b2582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:32.721922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1285/2588] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:32.730014 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.730299 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.730573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.739510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d538b-914a-4093-98df-e088e71b2582 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.743118 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.743728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.744001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.744267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.749465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:32.751264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:32.752906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:32.756671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.758706 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.761530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1286/2589] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:44:32.766321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.773281 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1304/2590] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:44:32.777137 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab8c9b-4c00-4c2a-a8a2-129b06f40699 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.781939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.782579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.782855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.783128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.783825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:32.784119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.785773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.798256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1287/2591] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:32.804371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.815770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b7f0-a491-4a50-bfe3-46935382ea22 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.823572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.824608 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.824926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.825229 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.825911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.828013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.861504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1305/2592] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:32 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:32.863009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.876544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f77cf3-9173-40a1-bc38-2c90f96755c8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.885408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:32.944081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.944704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.945006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.945272 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.945971 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.947876 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:32.969734 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1288/2593] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:32 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:32.974259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:32.985431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118215ee-5563-41c0-9ea9-76f3a5c2f7a5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:32.990643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:32.991263 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:32.991538 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:32.991806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:32.992504 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:32.994271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.008352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1306/2594] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:33.009611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.019666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49d13-c996-4649-8aaa-19163c5de347 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.024258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.025375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.027546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.027814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.028582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.031050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.043538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1289/2595] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:33.049385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.060838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4112ee-8822-48bd-ba68-fb4efdc7dec0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.061771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.063680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.067080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89cd99-5ab6-414c-9b2c-3a9eb1389fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.067786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89cd99-5ab6-414c-9b2c-3a9eb1389fec admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.068064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89cd99-5ab6-414c-9b2c-3a9eb1389fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.068329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89cd99-5ab6-414c-9b2c-3a9eb1389fec admin admin] PATH_INFO: `/v3/projects/6bffde7683ab4f75811bd142ebb77dfe/users/f697ff71139548cba4144f147b7e82df/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.076687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd409759-0bf0-435b-a7bd-533e4c1afe27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.082198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.082869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.083262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.083535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] PATH_INFO: `/v3/projects/fb653b1d1cf04fe8b27b2fb6004ee995` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.084933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb653b1d1cf04fe8b27b2fb6004ee995)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.085268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89cd99-5ab6-414c-9b2c-3a9eb1389fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bffde7683ab4f75811bd142ebb77dfe, user_id=f697ff71139548cba4144f147b7e82df, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.087057 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.088456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89cd99-5ab6-414c-9b2c-3a9eb1389fec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.105572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1307/2596] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:33 2022] PUT /identity/v3/projects/6bffde7683ab4f75811bd142ebb77dfe/users/f697ff71139548cba4144f147b7e82df/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:33.115772 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.116047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.116350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.147565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] MFA Rules not processed for user `f697ff71139548cba4144f147b7e82df`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:33.148953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.153913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb653b1d1cf04fe8b27b2fb6004ee995', 'request_id': 'req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.161116 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb653b1d1cf04fe8b27b2fb6004ee995', 'request_id': 'req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.166925 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb653b1d1cf04fe8b27b2fb6004ee995', 'request_id': 'req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.179184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] The token cache is being invalidate because project fb653b1d1cf04fe8b27b2fb6004ee995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:33.179508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.180128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.181697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1290/2597] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:33 2022] DELETE /identity/v3/projects/fb653b1d1cf04fe8b27b2fb6004ee995 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:33.185732 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.187111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.228919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1308/2598] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:44:33.232939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.234391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.261254 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab4fe0-563b-493c-b43a-f00d6ab04ba5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.265810 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.266462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.266725 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.266994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] PATH_INFO: `/v3/users/69921b470f7b4315b241802618903a5a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.268623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69921b470f7b4315b241802618903a5a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.270269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.284938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:33.311316 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b6cf8-f794-4343-93e8-7a1483f3b06a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.318465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.319080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.319355 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.319630 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.320311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.322251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.325111 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69921b470f7b4315b241802618903a5a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.333954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69921b470f7b4315b241802618903a5a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.334592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69921b470f7b4315b241802618903a5a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.347944 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69921b470f7b4315b241802618903a5a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.353119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1309/2599] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:33 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:33.354498 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.355310 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69921b470f7b4315b241802618903a5a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:33.364523 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1291/2600] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:33 2022] DELETE /identity/v3/users/69921b470f7b4315b241802618903a5a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:33.375270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:33.432750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641309-7a6f-4c7f-8d0d-66e11e59ba19 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.504052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.505221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.505221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.505649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.506310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.508298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.534136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1310/2601] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:33 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:33.538172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.539819 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.553208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b2c85-598a-470b-884c-ce9a24d36822 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.558687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.559354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.559630 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.559902 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.560601 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.562227 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.573051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1292/2602] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:33.578091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.588970 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27956f4-398f-459f-8b4c-558d126b4eb6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.593979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f8a46-1f77-40a0-90bd-705f988b975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.594598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f8a46-1f77-40a0-90bd-705f988b975a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.594879 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f8a46-1f77-40a0-90bd-705f988b975a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.595150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f8a46-1f77-40a0-90bd-705f988b975a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.595844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f8a46-1f77-40a0-90bd-705f988b975a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.597493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f8a46-1f77-40a0-90bd-705f988b975a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.608056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1311/2603] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:33.612361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.622609 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3ade-89f6-4043-8f50-cc6c2b8be0a4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.627487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.628171 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.628441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.628730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] PATH_INFO: `/v3/projects/ca9c753b63d14a87a967cd6c5f08ac36/users/c9a6f4725bf14b9c94d48b7ebbb60182/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.649373 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9c753b63d14a87a967cd6c5f08ac36, user_id=c9a6f4725bf14b9c94d48b7ebbb60182, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.652593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.669094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1293/2604] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:33 2022] PUT /identity/v3/projects/ca9c753b63d14a87a967cd6c5f08ac36/users/c9a6f4725bf14b9c94d48b7ebbb60182/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:33.680382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.680613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.682803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.717547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] MFA Rules not processed for user `c9a6f4725bf14b9c94d48b7ebbb60182`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:33.718964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.788542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1312/2605] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:33.793531 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.859936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee2ce5-61cd-4eca-a10a-1fd63a84cfbf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.865053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:33.865739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:33.866004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:33.866282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:33.866936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:33.868878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:33.902312 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1294/2606] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:33 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:33.906617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.926019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963417b-5aa9-4758-9b25-38f211a04b3c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:33.929326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:33.932551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:33.958483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:34.005467 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.006169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.006507 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.006842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.007318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:34.007643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.010307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.024413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998e95-3cc9-4865-8d18-5bdedea98032 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.031738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.032560 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.032982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.033263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] PATH_INFO: `/v3/projects/578630bf0d4f466b881c279a3b9a80ed` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.034688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578630bf0d4f466b881c279a3b9a80ed)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.036744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.050100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1313/2607] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:33 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:34.051133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.062547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361f957-5a0a-4a80-9180-5729d1c2ee93 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.067562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.068223 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.068477 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.068750 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.069503 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.071685 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.086377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1314/2608] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:34.091842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.102356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a1bd2-7d65-4eee-b7cf-da4b309c2a52 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.102619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578630bf0d4f466b881c279a3b9a80ed', 'request_id': 'req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:34.110385 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578630bf0d4f466b881c279a3b9a80ed', 'request_id': 'req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:34.116172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578630bf0d4f466b881c279a3b9a80ed', 'request_id': 'req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:34.129858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] The token cache is being invalidate because project 578630bf0d4f466b881c279a3b9a80ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:34.130215 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:34.130870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3365b09f-15a5-4ca3-bfb9-a4b31bc3cbe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:34.132813 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1295/2609] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:33 2022] DELETE /identity/v3/projects/578630bf0d4f466b881c279a3b9a80ed => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:34.142817 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa46fc2-f7a2-427e-89f8-cdf840221d41 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.143249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa46fc2-f7a2-427e-89f8-cdf840221d41 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.143519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa46fc2-f7a2-427e-89f8-cdf840221d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.170162 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4aa46fc2-f7a2-427e-89f8-cdf840221d41 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:34.171332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4aa46fc2-f7a2-427e-89f8-cdf840221d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.184003 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.184696 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.184998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.185481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] PATH_INFO: `/v3/projects/3335950b1dfc4d29bf65bb96e42d1ef6/users/d195a6029fe24dd7af9b23c47e6537a7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.200509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3335950b1dfc4d29bf65bb96e42d1ef6, user_id=d195a6029fe24dd7af9b23c47e6537a7, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.203273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.225300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1315/2610] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:34 2022] PUT /identity/v3/projects/3335950b1dfc4d29bf65bb96e42d1ef6/users/d195a6029fe24dd7af9b23c47e6537a7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:34.229329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.231242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.255864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1296/2611] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:34.264426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.264993 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.266108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.286938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:34.288174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.318172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082163c9-1f30-4981-8c52-489ad3c4fbbf admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.331301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.333221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.333595 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.333941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.334740 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.336735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.348419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1316/2612] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:34.352497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.364490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44c571-1f27-41a8-9082-3839c6b5ffe7 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.368797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1297/2613] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:34.369589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e432482-ef60-43ec-93ee-3d086342b02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.370293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.370528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e432482-ef60-43ec-93ee-3d086342b02d admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.370872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e432482-ef60-43ec-93ee-3d086342b02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.371207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e432482-ef60-43ec-93ee-3d086342b02d admin admin] PATH_INFO: `/v3/projects/3335950b1dfc4d29bf65bb96e42d1ef6/users/d195a6029fe24dd7af9b23c47e6537a7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.373449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e432482-ef60-43ec-93ee-3d086342b02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3335950b1dfc4d29bf65bb96e42d1ef6, user_id=d195a6029fe24dd7af9b23c47e6537a7, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.376539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e432482-ef60-43ec-93ee-3d086342b02d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.381193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd8b33-d5a5-422d-9c55-89f4c41f0dc2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.388299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1317/2614] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:34 2022] PUT /identity/v3/projects/3335950b1dfc4d29bf65bb96e42d1ef6/users/d195a6029fe24dd7af9b23c47e6537a7/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:34.400423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.400710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.401015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.405564 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:34.432396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:34.435881 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] MFA Rules not processed for user `d195a6029fe24dd7af9b23c47e6537a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:34.437780 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.463961 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.464623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.464935 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.465207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.465900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:34.466518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.469368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.486420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1298/2615] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:44:34.487594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.500933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedde6e-4978-4cd6-a96c-01d2cdad9a01 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.513738 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.515549 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.515918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.516258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.517454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.519691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.549310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1318/2616] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:34.554982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1299/2617] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:34 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:34.558999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.560442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.562083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.569596 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee530a-0c2a-4631-8833-d0152fef5d84 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.623201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a930e-9c23-44c5-a865-a70e5fe8ab3d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.626904 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.627881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.628154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.628420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.630864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.631480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.631749 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.632016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.632713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.634665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.655168 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1319/2618] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:34 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:34.659341 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.669108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d0e5e-4c55-462a-986e-ede1f490c37d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.673971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.674573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.674850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.675113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.675866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.677565 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.690358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1320/2619] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:34.694569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.696568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.698176 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.705158 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82fa18-8e74-4da5-9bba-9c1fa8b32ad3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.709842 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1300/2620] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:44:34.710382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc0b0e-3c76-45e9-aac4-9d0c9d47ce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.710996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0b0e-3c76-45e9-aac4-9d0c9d47ce72 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.711271 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0b0e-3c76-45e9-aac4-9d0c9d47ce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.711537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc0b0e-3c76-45e9-aac4-9d0c9d47ce72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.712227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc0b0e-3c76-45e9-aac4-9d0c9d47ce72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.713895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc0b0e-3c76-45e9-aac4-9d0c9d47ce72 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.726741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1321/2621] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:34.731141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.742067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d1c18-0cf1-475b-bc99-5be826f1e49a service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.746863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.747649 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.753344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.753799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] PATH_INFO: `/v3/projects/437dbf4d11be429eafaf3cb53518584b/users/36fba64144cf4aa0b12bd7a08902d415/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.773244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437dbf4d11be429eafaf3cb53518584b, user_id=36fba64144cf4aa0b12bd7a08902d415, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.776690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.797060 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1301/2622] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:34 2022] PUT /identity/v3/projects/437dbf4d11be429eafaf3cb53518584b/users/36fba64144cf4aa0b12bd7a08902d415/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:34.802165 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.802446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.802739 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.821335 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:34.822533 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.888170 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1322/2623] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:34.901844 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:34.903870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:34.917437 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:34.971451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c9bf9-9837-4910-9a1e-d07d88d3fc27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:34.975801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f89032-ab26-4887-9c4b-a9b23d9b5be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:34.976440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f89032-ab26-4887-9c4b-a9b23d9b5be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:34.976710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f89032-ab26-4887-9c4b-a9b23d9b5be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:34.977021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f89032-ab26-4887-9c4b-a9b23d9b5be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:34.987228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f89032-ab26-4887-9c4b-a9b23d9b5be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:34.988846 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f89032-ab26-4887-9c4b-a9b23d9b5be5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:34.998464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1302/2624] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:35.273497 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:35.284352 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:35.341085 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c5356-91ef-4dc3-b794-878b97faf254 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:35.345535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:35.346315 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:35.346691 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:35.347026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:35.349279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:35.426172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:35.427695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:35.445984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1323/2625] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:35.710789 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:35.711317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:35.711527 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:35.745947 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] MFA Rules not processed for user `36fba64144cf4aa0b12bd7a08902d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:35.747384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:35.843398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1303/2626] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:35.847959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:35.860582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64661e52-5a05-41b0-be6d-e3392d267b76 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:35.865545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:35.866370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:35.866818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:35.867183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:35.868054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:35.870404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:35.898712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1324/2627] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:35 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:35.902843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:35.914178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda216bd-3908-4438-b9bf-78fe57a5fec0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:35.977520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:35.978064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:35.978337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:35.978603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:35.979258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:35.981265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:36.007460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1304/2628] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:35 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:36.011893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:36.023297 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152ed15-b917-4e32-818b-bb49cb546735 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:36.028726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:36.029429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:36.029690 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:36.030002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:36.030817 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:36.032896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:36.045929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1325/2629] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:36.053656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:36.063214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b2b4e-990c-447b-901c-a9f9686cd9d4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:36.068948 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:36.069628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:36.069918 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:36.070207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] PATH_INFO: `/v3/projects/57482c086a0e461ca2cae96f06bf1631/users/7dfa381b03ee45af95e726df02bccb18/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:36.086966 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57482c086a0e461ca2cae96f06bf1631, user_id=7dfa381b03ee45af95e726df02bccb18, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:36.089936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:36.105459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1305/2630] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:36 2022] PUT /identity/v3/projects/57482c086a0e461ca2cae96f06bf1631/users/7dfa381b03ee45af95e726df02bccb18/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:44:36.109555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:36.119568 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcc590-6bd7-4188-9a9c-4f06e731939a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:36.176322 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928f820-f94a-424a-856a-8da9e052f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:36.176982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f820-f94a-424a-856a-8da9e052f346 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:36.177268 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f820-f94a-424a-856a-8da9e052f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:36.177535 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f820-f94a-424a-856a-8da9e052f346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:36.178219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f820-f94a-424a-856a-8da9e052f346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:36.179816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f820-f94a-424a-856a-8da9e052f346 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:36.190932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1326/2631] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:44:36.195493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:36.206663 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d63df4-693d-4ab6-bc07-9600f617c817 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:36.211784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:36.212448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:36.212724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:36.213028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] PATH_INFO: `/v3/projects/57482c086a0e461ca2cae96f06bf1631/users/7dfa381b03ee45af95e726df02bccb18/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:36.215173 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57482c086a0e461ca2cae96f06bf1631, user_id=7dfa381b03ee45af95e726df02bccb18, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:36.218191 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:36.221324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:36.230991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1306/2632] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:36 2022] PUT /identity/v3/projects/57482c086a0e461ca2cae96f06bf1631/users/7dfa381b03ee45af95e726df02bccb18/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:36.894490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:36.895559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:36.896087 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:36.926933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] MFA Rules not processed for user `7dfa381b03ee45af95e726df02bccb18`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:36.928184 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:36.942470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:37.008729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1327/2633] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:37.023256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:37.025306 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:37.101508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d144a3-2868-4393-853c-c4882c4c67e2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:37.105187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:37.105811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:37.106079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:37.106341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:37.162034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:37.163682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:37.176145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1307/2634] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:38.578486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:38.593567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefadc6-bbd1-45d7-9c88-b063b863cdf8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:38.598180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:38.598792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:38.599064 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:38.599331 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:38.609364 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:38.610808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:38.620946 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1328/2635] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:39.642199 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:39.651266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596149d3-24da-4615-abb4-f43632ed2d7e service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:39.654997 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:39.655605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:39.655882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:39.656154 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:39.657977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:39.730586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:39.732027 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07effb73-c760-4f1b-b7b4-15d5eddbea6b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:39.741443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1308/2636] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:41.922773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:41.993351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6797cb3-d087-4936-94fe-5aa8026117cd service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:41.998439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:41.999080 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:41.999357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:41.999662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] PATH_INFO: `/v3/users/f697ff71139548cba4144f147b7e82df` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:42.000914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f697ff71139548cba4144f147b7e82df)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:42.003200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:42.054617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f697ff71139548cba4144f147b7e82df'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.063016 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f697ff71139548cba4144f147b7e82df'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.063726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f697ff71139548cba4144f147b7e82df'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.070963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f697ff71139548cba4144f147b7e82df'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.071636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f697ff71139548cba4144f147b7e82df'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.089098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1329/2637] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:41 2022] DELETE /identity/v3/users/f697ff71139548cba4144f147b7e82df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:42.093882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ea460-90bf-422c-ad55-9236d060607b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:42.094387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ea460-90bf-422c-ad55-9236d060607b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:42.094387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ea460-90bf-422c-ad55-9236d060607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:42.113711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-206ea460-90bf-422c-ad55-9236d060607b None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:42.114895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-206ea460-90bf-422c-ad55-9236d060607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:42.162189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:42.183149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1309/2638] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:42.193509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:42.194959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:42.213609 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:42.257462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f40dcc-8ac9-4e13-995d-9f5d6d191deb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:42.261424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:42.262053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:42.262333 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:42.262586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:42.274415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:42.276470 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:42.287493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1330/2639] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:42.562121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ea460-90bf-422c-ad55-9236d060607b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:42.563654 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-206ea460-90bf-422c-ad55-9236d060607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:42.575943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ea460-90bf-422c-ad55-9236d060607b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:42.581327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:42.581949 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:42.582247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:42.582508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] PATH_INFO: `/v3/projects/6bffde7683ab4f75811bd142ebb77dfe` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:42.583770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bffde7683ab4f75811bd142ebb77dfe)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:42.585729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:42.639687 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bffde7683ab4f75811bd142ebb77dfe', 'request_id': 'req-be9d073d-ef69-4421-8353-84804b5d2ea1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.646125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bffde7683ab4f75811bd142ebb77dfe', 'request_id': 'req-be9d073d-ef69-4421-8353-84804b5d2ea1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.652980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bffde7683ab4f75811bd142ebb77dfe', 'request_id': 'req-be9d073d-ef69-4421-8353-84804b5d2ea1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.667297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] The token cache is being invalidate because project 6bffde7683ab4f75811bd142ebb77dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:42.667579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.668255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.669896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1310/2640] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:42 2022] DELETE /identity/v3/projects/6bffde7683ab4f75811bd142ebb77dfe => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:42.674207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:42.684425 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be43ae-5c36-481f-bd58-5d396726ffa8 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:42.744028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:42.744666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:42.744974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:42.745243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] PATH_INFO: `/v3/users/c9a6f4725bf14b9c94d48b7ebbb60182` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:42.746427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a6f4725bf14b9c94d48b7ebbb60182)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:42.748337 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:42.800555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a6f4725bf14b9c94d48b7ebbb60182'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.809328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a6f4725bf14b9c94d48b7ebbb60182'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.809887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a6f4725bf14b9c94d48b7ebbb60182'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.816986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a6f4725bf14b9c94d48b7ebbb60182'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.817604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a6f4725bf14b9c94d48b7ebbb60182'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:42.835501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1331/2641] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:42 2022] DELETE /identity/v3/users/c9a6f4725bf14b9c94d48b7ebbb60182 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:43.118936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:43.121047 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:43.135825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d073d-ef69-4421-8353-84804b5d2ea1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:43.191839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:43.192486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:43.192827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:43.193097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] PATH_INFO: `/v3/projects/ca9c753b63d14a87a967cd6c5f08ac36` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:43.194323 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9c753b63d14a87a967cd6c5f08ac36)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:43.196304 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:43.246330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9c753b63d14a87a967cd6c5f08ac36', 'request_id': 'req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.253498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9c753b63d14a87a967cd6c5f08ac36', 'request_id': 'req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.259186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9c753b63d14a87a967cd6c5f08ac36', 'request_id': 'req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.273217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] The token cache is being invalidate because project ca9c753b63d14a87a967cd6c5f08ac36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:43.273496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.274092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.275631 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1311/2642] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:43 2022] DELETE /identity/v3/projects/ca9c753b63d14a87a967cd6c5f08ac36 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:43.279766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:43.281348 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:43.294768 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371bb3-5cd9-4708-9ad0-b01f19e1698b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:43.333613 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:43.366511 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:43.367155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:43.367429 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:43.367702 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] PATH_INFO: `/v3/users/d195a6029fe24dd7af9b23c47e6537a7` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:43.368948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d195a6029fe24dd7af9b23c47e6537a7)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:43.370863 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:43.418705 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd195a6029fe24dd7af9b23c47e6537a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.426449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd195a6029fe24dd7af9b23c47e6537a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.427173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd195a6029fe24dd7af9b23c47e6537a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.434043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd195a6029fe24dd7af9b23c47e6537a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.434655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00cc1e90-20e8-427c-908a-1e6ac706ea9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd195a6029fe24dd7af9b23c47e6537a7'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.451372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1332/2643] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:43 2022] DELETE /identity/v3/users/d195a6029fe24dd7af9b23c47e6537a7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:43.734519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:43.736230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:43.748329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:43.757097 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27d4bd-8c9f-436d-a77f-09a0cfa5f2b2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:43.820367 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:43.821241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:43.821603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:43.821940 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] PATH_INFO: `/v3/projects/3335950b1dfc4d29bf65bb96e42d1ef6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:43.823762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3335950b1dfc4d29bf65bb96e42d1ef6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:43.826797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:43.879320 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3335950b1dfc4d29bf65bb96e42d1ef6', 'request_id': 'req-f31566c9-9e01-40dd-9819-e3248db721b5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.885493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3335950b1dfc4d29bf65bb96e42d1ef6', 'request_id': 'req-f31566c9-9e01-40dd-9819-e3248db721b5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.891652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3335950b1dfc4d29bf65bb96e42d1ef6', 'request_id': 'req-f31566c9-9e01-40dd-9819-e3248db721b5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.905493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] The token cache is being invalidate because project 3335950b1dfc4d29bf65bb96e42d1ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:43.905840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.906528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f31566c9-9e01-40dd-9819-e3248db721b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:43.908050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1312/2644] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:43 2022] DELETE /identity/v3/projects/3335950b1dfc4d29bf65bb96e42d1ef6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:43.914154 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:43.914436 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:43.914704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:43.934273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:43.935462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:43.996328 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1333/2645] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:44.005332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.005613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.005885 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.027572 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:44.028793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:44.041801 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1313/2646] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:44.043520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.054604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ce341-c43f-470e-82a0-cfab96f17eee None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.059766 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.060375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.060647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.064920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.064920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:44.064920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.064920 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.075919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1334/2647] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:44.080442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.089764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040e8fb-8c49-4f11-9be8-bde26c364663 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.095599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.096201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.096472 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.096735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.097430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.099318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.135843 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1314/2648] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:44.140586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.151013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11fe8c-248b-4cd8-8244-7f6eb3f3a192 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.194794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:44.212483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.213133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.213405 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.213671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.214329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.216228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.239549 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1335/2649] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:44.243929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.255442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84cee3-24aa-4c9b-9bb7-c9ffba83c75c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.260502 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.261152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.261432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.261702 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.262377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.264041 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.275251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1315/2650] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:44.279983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.295517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ebb36-870e-422d-86e4-b906b5cdbdd5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.301975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f78efd-9a7f-4cc3-bbab-aa41c3efb64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.302591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f78efd-9a7f-4cc3-bbab-aa41c3efb64d admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.302866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f78efd-9a7f-4cc3-bbab-aa41c3efb64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.303133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f78efd-9a7f-4cc3-bbab-aa41c3efb64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.303826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f78efd-9a7f-4cc3-bbab-aa41c3efb64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.305473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f78efd-9a7f-4cc3-bbab-aa41c3efb64d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.316854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1336/2651] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:44.321234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.331520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dd9a3-f640-4da9-b3ea-c2473b7205f0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.337249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.337924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.338194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.338436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] PATH_INFO: `/v3/projects/0dc7d491866943ddab1483235e307f41/users/74786db00a5545b089aa0f3d32062be9/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.354305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc7d491866943ddab1483235e307f41, user_id=74786db00a5545b089aa0f3d32062be9, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.357000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.372740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1316/2652] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:44 2022] PUT /identity/v3/projects/0dc7d491866943ddab1483235e307f41/users/74786db00a5545b089aa0f3d32062be9/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:44.386531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.386764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.387213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.418104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] MFA Rules not processed for user `74786db00a5545b089aa0f3d32062be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:44.419353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:44.485516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1337/2653] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:44.490149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.521567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:44.556510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbfc2-a676-49fb-88e2-790545119602 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.562316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.562973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.563249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.563516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.564179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.566156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.598783 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1317/2654] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:44.602910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.615055 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8040c-9865-4716-b064-6f56fcded73e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.674684 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.675324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.675599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.675868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.676519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.678513 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.702044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1338/2655] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:44.706112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.716623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc5d6-c8b6-4501-9d64-fc9a1147bd26 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.721709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.722371 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.722637 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.722899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.723586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.725316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.736474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1318/2656] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:44.740546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.751226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8709faa-a835-4039-bd68-018a2b05a858 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.757421 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.758071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.758350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.758624 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] PATH_INFO: `/v3/projects/8d4313351cd74491b0df0c25ef224717/users/2252af11ef334ea6a86ad1a739e18955/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.775194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4313351cd74491b0df0c25ef224717, user_id=2252af11ef334ea6a86ad1a739e18955, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.778001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.793338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1339/2657] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:44 2022] PUT /identity/v3/projects/8d4313351cd74491b0df0c25ef224717/users/2252af11ef334ea6a86ad1a739e18955/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:44.797678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.807833 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ea4bf-2e88-431b-9af1-f76fc3dd1fb5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.865479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11eab9e-2a66-4eca-b6ff-4275a3b4d7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.866139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eab9e-2a66-4eca-b6ff-4275a3b4d7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.866396 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eab9e-2a66-4eca-b6ff-4275a3b4d7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.866662 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eab9e-2a66-4eca-b6ff-4275a3b4d7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.867351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eab9e-2a66-4eca-b6ff-4275a3b4d7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.869030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eab9e-2a66-4eca-b6ff-4275a3b4d7d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.879707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1319/2658] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:44.884052 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:44.894719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c08aa-5987-4e5b-b1f2-d3d940cb6c68 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:44.900161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:44.901026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.901383 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.901703 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] PATH_INFO: `/v3/projects/8d4313351cd74491b0df0c25ef224717/users/2252af11ef334ea6a86ad1a739e18955/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:44.904006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4313351cd74491b0df0c25ef224717, user_id=2252af11ef334ea6a86ad1a739e18955, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:44.906729 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:44.917800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1340/2659] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:44 2022] PUT /identity/v3/projects/8d4313351cd74491b0df0c25ef224717/users/2252af11ef334ea6a86ad1a739e18955/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:44.990719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:44.990923 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:44.991143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:45.019387 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] MFA Rules not processed for user `2252af11ef334ea6a86ad1a739e18955`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:45.020651 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:45.097614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1320/2660] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:45.108742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:45.110560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:45.116963 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:45.171983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e7593-8dd7-404a-a070-0b1cf7f833df admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:45.175632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:45.176250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:45.176522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:45.176812 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:45.185512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:45.187349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:45.199527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1341/2661] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:45.280151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:45.292521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc21e84-728f-4bc3-88b2-9a067feb64e0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:45.296672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d169181-373e-4dab-9507-ad0eb3d759b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:45.297393 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d169181-373e-4dab-9507-ad0eb3d759b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:45.297894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d169181-373e-4dab-9507-ad0eb3d759b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:45.298233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d169181-373e-4dab-9507-ad0eb3d759b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:45.320998 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:45.356909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d169181-373e-4dab-9507-ad0eb3d759b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:45.358344 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d169181-373e-4dab-9507-ad0eb3d759b5 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:45.367039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1321/2662] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:55.193912 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=24 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:55.363330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=17 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:55.811171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:55.876622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72155e7-d350-406a-9a65-285541eefa48 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:55.881974 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:55.882643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:55.882949 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:55.883220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] PATH_INFO: `/v3/users/74786db00a5545b089aa0f3d32062be9` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:55.884418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=74786db00a5545b089aa0f3d32062be9)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:55.886395 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:55.935649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74786db00a5545b089aa0f3d32062be9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:55.943078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74786db00a5545b089aa0f3d32062be9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:55.943682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74786db00a5545b089aa0f3d32062be9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:55.953773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74786db00a5545b089aa0f3d32062be9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:55.954500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74786db00a5545b089aa0f3d32062be9'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:55.977543 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1342/2663] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:55 2022] DELETE /identity/v3/users/74786db00a5545b089aa0f3d32062be9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:55.983155 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:55.983506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:55.983868 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:56.009892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:56.011131 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:56.083442 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1322/2664] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:56.093732 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:56.095460 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:56.161642 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4e2bf-d55a-452d-a88b-35a432523cae admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:56.166810 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:56.167282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:56.167553 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:56.167816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:56.183767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:56.183767 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:56.189420 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1343/2665] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:44:56.611463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:56.613042 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:56.630118 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9ad78-2eb3-4661-9c55-0128cfa82fc4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:56.638434 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:56.639356 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:56.639955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:56.640231 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] PATH_INFO: `/v3/projects/0dc7d491866943ddab1483235e307f41` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:56.641507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc7d491866943ddab1483235e307f41)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:56.650124 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:56.703005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc7d491866943ddab1483235e307f41', 'request_id': 'req-e64dd28d-5e23-4c1f-8a04-51503e335ce1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.709441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc7d491866943ddab1483235e307f41', 'request_id': 'req-e64dd28d-5e23-4c1f-8a04-51503e335ce1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.714873 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc7d491866943ddab1483235e307f41', 'request_id': 'req-e64dd28d-5e23-4c1f-8a04-51503e335ce1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.728507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] The token cache is being invalidate because project 0dc7d491866943ddab1483235e307f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:56.728809 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.729422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.730941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1323/2666] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:56 2022] DELETE /identity/v3/projects/0dc7d491866943ddab1483235e307f41 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:56.735044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:56.745330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e88675-96a6-4e79-87b2-4edeaa5fb4de service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:56.765672 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:56.812207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:56.812927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:56.813207 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:56.813492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] PATH_INFO: `/v3/users/2252af11ef334ea6a86ad1a739e18955` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:56.814676 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2252af11ef334ea6a86ad1a739e18955)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:56.816634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:56.865018 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2252af11ef334ea6a86ad1a739e18955'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.873211 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2252af11ef334ea6a86ad1a739e18955'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.874011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2252af11ef334ea6a86ad1a739e18955'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.881361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2252af11ef334ea6a86ad1a739e18955'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.881998 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-56c3634c-0eea-4157-abd4-54933fa79574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2252af11ef334ea6a86ad1a739e18955'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:56.901114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1344/2667] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:44:56 2022] DELETE /identity/v3/users/2252af11ef334ea6a86ad1a739e18955 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:57.185605 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.187550 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:57.202530 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dd28d-5e23-4c1f-8a04-51503e335ce1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.261301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.261837 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.262112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.262377 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] PATH_INFO: `/v3/projects/8d4313351cd74491b0df0c25ef224717` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.263545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4313351cd74491b0df0c25ef224717)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.265506 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.297991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:57.315845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4313351cd74491b0df0c25ef224717', 'request_id': 'req-fbb7266c-5539-4310-9182-204be6243891'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:57.321883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4313351cd74491b0df0c25ef224717', 'request_id': 'req-fbb7266c-5539-4310-9182-204be6243891'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:57.327973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4313351cd74491b0df0c25ef224717', 'request_id': 'req-fbb7266c-5539-4310-9182-204be6243891'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:57.342305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] The token cache is being invalidate because project 8d4313351cd74491b0df0c25ef224717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:44:57.342590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:57.343275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fbb7266c-5539-4310-9182-204be6243891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:44:57.345021 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1324/2668] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:44:57 2022] DELETE /identity/v3/projects/8d4313351cd74491b0df0c25ef224717 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:57.350166 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.350448 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.350720 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.370049 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:57.371248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:57.433933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1345/2669] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:57.442040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.442321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.442586 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.463139 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:57.464358 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:57.474249 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1325/2670] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:57.478594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.488900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9362a61-ed49-4903-b3d9-820ac1533f9e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.493792 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.494400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.494682 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.494959 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.495651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:44:57.495923 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.497585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.510650 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1346/2671] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:44:57.515628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.525423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43fd85-f78c-4cf4-8bc3-fef69841cc6f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.530457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.531064 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.531336 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.531625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.532280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.534206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.564362 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1326/2672] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:44:57 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:44:57.568743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.579582 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebe041-93db-4a6b-bf62-167cb0618403 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.615113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:44:57.668708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.669053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.669380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.669709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.673515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.677368 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.705029 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1347/2673] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:44:57 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:44:57.709046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.725771 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba48bcc-9d19-4ea9-8664-bd01aef4d844 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.737360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.737999 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.738275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.738544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.739245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.740865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.779484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1327/2674] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:57.784737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.798617 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e1fc-30f8-4d8a-b9cb-bdb55b0292be admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.812002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f321dc-1035-4882-9202-99d60a0f51ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.813686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f321dc-1035-4882-9202-99d60a0f51ab admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.815314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f321dc-1035-4882-9202-99d60a0f51ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.820967 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f321dc-1035-4882-9202-99d60a0f51ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.821273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f321dc-1035-4882-9202-99d60a0f51ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.823041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f321dc-1035-4882-9202-99d60a0f51ab admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.835741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1348/2675] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:44:57.839692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:57.855539 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948609c8-0a29-48ee-9405-4404312731e1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:57.866293 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:57.867100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.867245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.867517 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] PATH_INFO: `/v3/projects/86cdcaf16d9c4ccc81943acca6817439/users/8c5acd06c6404531a92b93a075278ae5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.890719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cdcaf16d9c4ccc81943acca6817439, user_id=8c5acd06c6404531a92b93a075278ae5, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:57.893555 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:57.913321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1328/2676] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:44:57 2022] PUT /identity/v3/projects/86cdcaf16d9c4ccc81943acca6817439/users/8c5acd06c6404531a92b93a075278ae5/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:44:57.927724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:57.928014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:57.928808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:57.960426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] MFA Rules not processed for user `8c5acd06c6404531a92b93a075278ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:44:57.961868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:58.035159 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1349/2677] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:44:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:44:58.047613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:44:58.049449 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:44:58.107700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda7052-7890-4f36-868a-c9166c1bcada admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:44:58.111576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:44:58.112300 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:44:58.112658 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:44:58.113034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:44:58.121854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:44:58.123379 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:44:58.133565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1329/2678] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:44:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:02.641629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:02.643227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:02.705914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb596e-0d07-419e-8c7d-5a04f7fc6a9e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:02.711683 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:02.712339 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:02.712610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:02.712910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] PATH_INFO: `/v3/users/36fba64144cf4aa0b12bd7a08902d415` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:02.714199 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36fba64144cf4aa0b12bd7a08902d415)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:02.716125 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:02.770861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:02.777983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36fba64144cf4aa0b12bd7a08902d415'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:02.786382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36fba64144cf4aa0b12bd7a08902d415'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:02.787030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36fba64144cf4aa0b12bd7a08902d415'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:02.794189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36fba64144cf4aa0b12bd7a08902d415'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:02.794811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36fba64144cf4aa0b12bd7a08902d415'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:02.812280 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1350/2679] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:02 2022] DELETE /identity/v3/users/36fba64144cf4aa0b12bd7a08902d415 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:02.995792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.008534 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c9df1-25e2-48ad-83e3-009e9765c730 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.061460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:03.073217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.074095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.074471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.074926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] PATH_INFO: `/v3/projects/437dbf4d11be429eafaf3cb53518584b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.076348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437dbf4d11be429eafaf3cb53518584b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:03.078297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:03.127526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437dbf4d11be429eafaf3cb53518584b', 'request_id': 'req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.133736 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437dbf4d11be429eafaf3cb53518584b', 'request_id': 'req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.139554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437dbf4d11be429eafaf3cb53518584b', 'request_id': 'req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.153675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] The token cache is being invalidate because project 437dbf4d11be429eafaf3cb53518584b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:03.153952 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.154579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.156088 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1330/2680] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:02 2022] DELETE /identity/v3/projects/437dbf4d11be429eafaf3cb53518584b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:03.160397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.161902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:03.175039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d449ff7-c2c1-4a96-89a2-b046a13e823c admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.242090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.242738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.243015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.243289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] PATH_INFO: `/v3/users/7dfa381b03ee45af95e726df02bccb18` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.244530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfa381b03ee45af95e726df02bccb18)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:03.246493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:03.296356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfa381b03ee45af95e726df02bccb18'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.304450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfa381b03ee45af95e726df02bccb18'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.305071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfa381b03ee45af95e726df02bccb18'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.311856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfa381b03ee45af95e726df02bccb18'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.312485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-00c27860-91f8-4f67-8655-660ce8ab3cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfa381b03ee45af95e726df02bccb18'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.329848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1351/2681] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:03 2022] DELETE /identity/v3/users/7dfa381b03ee45af95e726df02bccb18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:03.508613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.510119 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:03.522751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c20d4-90ff-4682-80bd-c9eb36e4c9d0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.577677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.578243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.578548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.578780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] PATH_INFO: `/v3/projects/57482c086a0e461ca2cae96f06bf1631` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.579870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57482c086a0e461ca2cae96f06bf1631)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:03.581863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:03.630992 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57482c086a0e461ca2cae96f06bf1631', 'request_id': 'req-b84009ca-7d8c-4acf-830c-a0b470f5fd97'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.637081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57482c086a0e461ca2cae96f06bf1631', 'request_id': 'req-b84009ca-7d8c-4acf-830c-a0b470f5fd97'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.643324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57482c086a0e461ca2cae96f06bf1631', 'request_id': 'req-b84009ca-7d8c-4acf-830c-a0b470f5fd97'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.658660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] The token cache is being invalidate because project 57482c086a0e461ca2cae96f06bf1631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:03.658989 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.659933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b84009ca-7d8c-4acf-830c-a0b470f5fd97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:03.663113 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1331/2682] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:03 2022] DELETE /identity/v3/projects/57482c086a0e461ca2cae96f06bf1631 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:03.668321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.668857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.668857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.689143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:03.690323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:03.719731 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:03.748424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1352/2683] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:03.756962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.757244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.757509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.777941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:03.779151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:03.787922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1332/2684] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:03.792150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.802093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36488a40-2ac6-4f2d-8b81-b92cf72121ba None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.807093 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.807688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.807958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.808220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.808956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:03.809251 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:03.810845 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:03.822358 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1353/2685] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:45:03.826640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.836296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018aa80-87cf-4158-b2ba-5a131b95fe55 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.842166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.842759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.843029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.843289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.843937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:03.845915 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:03.850626 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:03.877214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1333/2686] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:03 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:03.881289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.892518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f466d-d2f0-4b52-8ff5-6ead28288ee2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.950456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.951071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.951350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.951607 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:03.952256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:03.954195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:03.978098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1354/2687] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:03 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:03.982282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:03.993258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15268ada-f454-4460-a76e-72bdbb8176c9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:03.998603 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:03.999235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:03.999484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:03.999755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.000453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.002520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:04.012982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1334/2688] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:04.016953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:04.027519 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485f2b7-5828-453b-bfee-022cc98567a3 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:04.033495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346feaaf-d528-4960-bf7e-39ce2840d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:04.034107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346feaaf-d528-4960-bf7e-39ce2840d6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.034422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346feaaf-d528-4960-bf7e-39ce2840d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.034924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346feaaf-d528-4960-bf7e-39ce2840d6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.035414 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346feaaf-d528-4960-bf7e-39ce2840d6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.037130 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346feaaf-d528-4960-bf7e-39ce2840d6d9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:04.048418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1355/2689] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:04.052730 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:04.063363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfe57-361c-4fed-96bb-e3878a07320d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:04.068453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:04.069144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.069419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.069690 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] PATH_INFO: `/v3/projects/7ab03d2805b74d2b925dbb6857f1cd0a/users/e3708b9444fd4d80a3e0acd8a3dc6273/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.085782 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab03d2805b74d2b925dbb6857f1cd0a, user_id=e3708b9444fd4d80a3e0acd8a3dc6273, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.088608 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:04.103573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1335/2690] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:04 2022] PUT /identity/v3/projects/7ab03d2805b74d2b925dbb6857f1cd0a/users/e3708b9444fd4d80a3e0acd8a3dc6273/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:04.115063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.115357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.115629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.144803 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] MFA Rules not processed for user `e3708b9444fd4d80a3e0acd8a3dc6273`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:04.146032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:04.215127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1356/2691] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:04.226516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:04.228403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:04.285735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532237b-974b-44db-9271-83849fd62536 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:04.289840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f6751-9817-4d3d-9384-d0cf9338364f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:04.290454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6751-9817-4d3d-9384-d0cf9338364f service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.290727 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6751-9817-4d3d-9384-d0cf9338364f service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.290996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6751-9817-4d3d-9384-d0cf9338364f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.300202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f6751-9817-4d3d-9384-d0cf9338364f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.301666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f6751-9817-4d3d-9384-d0cf9338364f service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:04.310753 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1336/2692] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:04.330334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:04.366944 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:04.408614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246d8df-ceda-4277-8869-c9b35b81b94f None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:04.413686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:04.414350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.414622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.414913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] PATH_INFO: `/v3/users/e3708b9444fd4d80a3e0acd8a3dc6273` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.416112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3708b9444fd4d80a3e0acd8a3dc6273)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.418090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:04.466581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3708b9444fd4d80a3e0acd8a3dc6273'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:04.474469 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3708b9444fd4d80a3e0acd8a3dc6273'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:04.475147 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3708b9444fd4d80a3e0acd8a3dc6273'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:04.482331 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3708b9444fd4d80a3e0acd8a3dc6273'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:04.482971 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3708b9444fd4d80a3e0acd8a3dc6273'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:04.500800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1357/2693] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:04 2022] DELETE /identity/v3/users/e3708b9444fd4d80a3e0acd8a3dc6273 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:04.505907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.506185 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.506460 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.526050 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:04.527238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:04.589451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1337/2694] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:04.600181 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:04.601764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:04.660357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8ea30-d3dc-4c69-b780-c2e27fd4f98d admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:04.664347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6882905-1ab7-4cbe-97f1-b056d07b130a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:04.664990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882905-1ab7-4cbe-97f1-b056d07b130a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.665266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882905-1ab7-4cbe-97f1-b056d07b130a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.665527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882905-1ab7-4cbe-97f1-b056d07b130a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.675522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882905-1ab7-4cbe-97f1-b056d07b130a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.676994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6882905-1ab7-4cbe-97f1-b056d07b130a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:04.686598 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1358/2695] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:04.966425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:04.967850 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:04.980852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656759a-4ecb-42c5-bbdf-0b5935725859 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:04.985451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:04.986084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:04.986478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:04.986698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] PATH_INFO: `/v3/projects/7ab03d2805b74d2b925dbb6857f1cd0a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:04.988860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab03d2805b74d2b925dbb6857f1cd0a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:04.994205 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.041210 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:05.042759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab03d2805b74d2b925dbb6857f1cd0a', 'request_id': 'req-5a086bf2-52d7-4e52-80df-c48b52e989b8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:05.051660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab03d2805b74d2b925dbb6857f1cd0a', 'request_id': 'req-5a086bf2-52d7-4e52-80df-c48b52e989b8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:05.057516 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab03d2805b74d2b925dbb6857f1cd0a', 'request_id': 'req-5a086bf2-52d7-4e52-80df-c48b52e989b8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:05.070744 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] The token cache is being invalidate because project 7ab03d2805b74d2b925dbb6857f1cd0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:05.071076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:05.071724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a086bf2-52d7-4e52-80df-c48b52e989b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:05.073450 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1338/2696] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:04 2022] DELETE /identity/v3/projects/7ab03d2805b74d2b925dbb6857f1cd0a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:05.078755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.079056 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.079572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.106403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:05.107723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:05.160574 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:05.175804 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1359/2697] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:05.187834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.188001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.188269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.208174 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:05.209422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:05.225562 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1339/2698] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:05.230069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.240193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726ff9c-c34a-4fbb-bc16-53779538d1a4 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.245194 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.245825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.246098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.246366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.247096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:05.247389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.249024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.264925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1360/2699] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:45:05.266103 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.275546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d2071-3b2e-432d-97ef-060ff3a3fca0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.281187 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.282005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.282278 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.282648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.283333 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.285907 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.317514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1340/2700] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:05 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:05.321984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.335253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2a3a2-e836-41e6-8fdf-8200b9a7a067 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.393788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.394631 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.395133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.395587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.396787 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.399086 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.428371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1361/2701] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:05 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:05.432674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.448498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5204868-0c5a-492a-b51b-134106867262 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.453706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.454422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.454620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.454852 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.455544 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.457166 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.467737 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1341/2702] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:05.472489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.491248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f3ea5-7e49-4716-95c7-9267996fc2f2 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.502591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b30c05-aed1-40d7-819b-a131ee2700a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.503248 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b30c05-aed1-40d7-819b-a131ee2700a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.503586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b30c05-aed1-40d7-819b-a131ee2700a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.503866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b30c05-aed1-40d7-819b-a131ee2700a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.504828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b30c05-aed1-40d7-819b-a131ee2700a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.506799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b30c05-aed1-40d7-819b-a131ee2700a5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.519527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1362/2703] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:45:05.523962 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.534731 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823c239-3186-4e70-a3c8-e83cc8b12cf8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.539645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.540440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.540851 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.541794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] PATH_INFO: `/v3/projects/4f0001bd61ec499395c7989d7eb804f0/users/bcb2733aec624f72badfa4a5384ab295/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.558671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0001bd61ec499395c7989d7eb804f0, user_id=bcb2733aec624f72badfa4a5384ab295, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.562169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.588074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1342/2704] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:05 2022] PUT /identity/v3/projects/4f0001bd61ec499395c7989d7eb804f0/users/bcb2733aec624f72badfa4a5384ab295/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:05.590074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.590380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.590655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.618806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] MFA Rules not processed for user `bcb2733aec624f72badfa4a5384ab295`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:05.620091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:05.686189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1363/2705] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:05.701739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:05.703855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:05.770277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e63ca-0635-4102-b4eb-fae519b38ade admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:05.774634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce26a0-5ab0-4a62-8330-a0ea553f1313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:05.775405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce26a0-5ab0-4a62-8330-a0ea553f1313 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:05.775718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce26a0-5ab0-4a62-8330-a0ea553f1313 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:05.775946 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce26a0-5ab0-4a62-8330-a0ea553f1313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:05.785773 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce26a0-5ab0-4a62-8330-a0ea553f1313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:05.787409 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce26a0-5ab0-4a62-8330-a0ea553f1313 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:05.796903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1343/2706] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:45:09.889586 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:09.893591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:09.947799 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:09.965346 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65a282-0c59-40b1-9210-27bf2b609419 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:09.971112 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:09.971764 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:09.972038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:09.972304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] PATH_INFO: `/v3/users/8c5acd06c6404531a92b93a075278ae5` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:09.973712 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5acd06c6404531a92b93a075278ae5)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:09.975649 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:10.025432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5acd06c6404531a92b93a075278ae5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.033774 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5acd06c6404531a92b93a075278ae5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.034392 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5acd06c6404531a92b93a075278ae5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.042063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5acd06c6404531a92b93a075278ae5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.042754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5acd06c6404531a92b93a075278ae5'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.060014 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1364/2707] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:09 2022] DELETE /identity/v3/users/8c5acd06c6404531a92b93a075278ae5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:10.064569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.065102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.065102 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.083905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:10.085140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:10.144446 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1344/2708] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:10.154371 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:10.155855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:10.216674 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f57dd-b3b0-4eb0-a8d3-2bca1ecb2d21 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:10.220417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a0f31-9f7c-4932-881b-07638a5d20b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:10.221068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0f31-9f7c-4932-881b-07638a5d20b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.221347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0f31-9f7c-4932-881b-07638a5d20b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.221619 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0f31-9f7c-4932-881b-07638a5d20b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.232800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a0f31-9f7c-4932-881b-07638a5d20b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:10.234240 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a0f31-9f7c-4932-881b-07638a5d20b7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:10.244238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1365/2709] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:10.517754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:10.519289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:10.531676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd99a3-a32f-4537-a798-277f762dc451 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:10.543638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:10.544668 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.545221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.545638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] PATH_INFO: `/v3/projects/86cdcaf16d9c4ccc81943acca6817439` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.547178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86cdcaf16d9c4ccc81943acca6817439)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:10.549066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:10.560579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:10.603629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86cdcaf16d9c4ccc81943acca6817439', 'request_id': 'req-49087bc6-2f9c-43d1-8943-2ce9f76463e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.609954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86cdcaf16d9c4ccc81943acca6817439', 'request_id': 'req-49087bc6-2f9c-43d1-8943-2ce9f76463e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.615932 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86cdcaf16d9c4ccc81943acca6817439', 'request_id': 'req-49087bc6-2f9c-43d1-8943-2ce9f76463e1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.642767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] The token cache is being invalidate because project 86cdcaf16d9c4ccc81943acca6817439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:10.643049 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.643710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49087bc6-2f9c-43d1-8943-2ce9f76463e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:10.645403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1345/2710] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:10 2022] DELETE /identity/v3/projects/86cdcaf16d9c4ccc81943acca6817439 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:10.651592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.651835 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.652164 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.724686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:10.725941 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:10.753171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:10.832367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1366/2711] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:10.844274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.844528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.844818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.865692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:10.866914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:10.876330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1346/2712] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:10.881274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:10.891667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45eea-4c00-4970-bbaf-4f40b261171c None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:10.897849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:10.898388 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.898660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.898919 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.899615 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:10.899899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:10.901539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:10.913730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1367/2713] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:45:10.917920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:10.927266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add79aba-6a9e-4710-8083-586a14d79682 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:10.932245 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:10.932875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:10.933153 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:10.933416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:10.934082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:10.935964 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:10.965847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1347/2714] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:10 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:10.970247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:10.981131 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36367d-2ad1-4010-bc6c-f94d6ad16e70 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:11.042722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:11.043351 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:11.043626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:11.043896 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:11.044558 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:11.046530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:11.073575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1368/2715] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:10 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:11.077712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:11.088466 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8c8ba-9bb9-40bf-a9ba-323440608d61 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:11.093832 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:11.094444 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:11.094719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:11.094982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:11.095722 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:11.097478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:11.108451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1348/2716] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:11.112564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:11.122754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754dd46-17ce-4ab2-82ef-f1a719a04368 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:11.127989 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fed4b-d3c8-4627-b13f-d102b4f939ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:11.128587 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fed4b-d3c8-4627-b13f-d102b4f939ef admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:11.128888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fed4b-d3c8-4627-b13f-d102b4f939ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:11.129155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fed4b-d3c8-4627-b13f-d102b4f939ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:11.129838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fed4b-d3c8-4627-b13f-d102b4f939ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:11.131411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fed4b-d3c8-4627-b13f-d102b4f939ef admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:11.142077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1369/2717] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:11.146381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:11.156691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c01b4-caa7-431c-b7d0-7ec2f1792e48 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:11.162529 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:11.163203 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:11.163471 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:11.163739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] PATH_INFO: `/v3/projects/6a503222f5964076a75d291a6530097c/users/c7dd082069b3423a8686cd809c8d4110/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:11.178232 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a503222f5964076a75d291a6530097c, user_id=c7dd082069b3423a8686cd809c8d4110, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:11.180912 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:11.195536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1349/2718] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:11 2022] PUT /identity/v3/projects/6a503222f5964076a75d291a6530097c/users/c7dd082069b3423a8686cd809c8d4110/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:11.207857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c4f8-ebda-42ae-bf3a-ab7cef379636 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:11.208187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c4f8-ebda-42ae-bf3a-ab7cef379636 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:11.208442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c4f8-ebda-42ae-bf3a-ab7cef379636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:11.237850 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c0a3c4f8-ebda-42ae-bf3a-ab7cef379636 None None] MFA Rules not processed for user `c7dd082069b3423a8686cd809c8d4110`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:11.239129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c0a3c4f8-ebda-42ae-bf3a-ab7cef379636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:11.283811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:11.306459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1370/2719] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:11.317433 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:11.319329 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:11.392877 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91c206-e437-447a-8a0c-bccedeb595f7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:11.397294 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:11.397898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:11.398201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:11.398462 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:11.407353 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:11.408703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:11.417770 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1350/2720] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:12.794192 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf7f3-ab7a-4055-96fb-965f5b4812a2 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:12.794741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf7f3-ab7a-4055-96fb-965f5b4812a2 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:12.794837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cf7f3-ab7a-4055-96fb-965f5b4812a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:12.815851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c91cf7f3-ab7a-4055-96fb-965f5b4812a2 None None] MFA Rules not processed for user `c7dd082069b3423a8686cd809c8d4110`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:12.817270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c91cf7f3-ab7a-4055-96fb-965f5b4812a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:12.825176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1371/2721] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:13.163052 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:13.164691 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:13.179656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:13.236804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f496e-f9ce-4941-bfb9-963aabc6a722 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:13.241548 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:13.242218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:13.242496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:13.242764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] PATH_INFO: `/v3/users/c9c5d3fa5a4b45d79ea7f6fd5dde2997` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:13.243936 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c5d3fa5a4b45d79ea7f6fd5dde2997)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:13.245943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:13.298497 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c5d3fa5a4b45d79ea7f6fd5dde2997'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.308059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c5d3fa5a4b45d79ea7f6fd5dde2997'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.308587 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c5d3fa5a4b45d79ea7f6fd5dde2997'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.324507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c5d3fa5a4b45d79ea7f6fd5dde2997'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.332676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c5d3fa5a4b45d79ea7f6fd5dde2997'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.339765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1351/2722] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:13 2022] DELETE /identity/v3/users/c9c5d3fa5a4b45d79ea7f6fd5dde2997 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:13.344077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:13.344603 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:13.345078 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:13.368320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:13.369520 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:13.429084 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1372/2723] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:13.439507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:13.441129 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:13.501167 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41dc45-e56e-4fc4-b1a9-4dc0b916f339 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:13.505246 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dc25d-1d24-4649-8357-8bb218450f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:13.505898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dc25d-1d24-4649-8357-8bb218450f5a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:13.506165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dc25d-1d24-4649-8357-8bb218450f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:13.506428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dc25d-1d24-4649-8357-8bb218450f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:13.516109 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dc25d-1d24-4649-8357-8bb218450f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:13.517592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dc25d-1d24-4649-8357-8bb218450f5a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:13.527438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1352/2724] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:45:13.875589 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:13.878632 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:13.898160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c80a-822c-4a8a-97a6-a3cda4e7a2b3 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:13.904599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:13.913230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:13.913510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:13.913956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] PATH_INFO: `/v3/projects/a5fed4f0efcc4c7184029437668e4286` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:13.915496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fed4f0efcc4c7184029437668e4286)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:13.917509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:13.975282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fed4f0efcc4c7184029437668e4286', 'request_id': 'req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.982583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fed4f0efcc4c7184029437668e4286', 'request_id': 'req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:13.988992 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fed4f0efcc4c7184029437668e4286', 'request_id': 'req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:14.005439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] The token cache is being invalidate because project a5fed4f0efcc4c7184029437668e4286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:14.005727 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:14.006637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ca754d61-1a5f-42ba-84a8-cd7fa73fbdb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:14.008174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1373/2725] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:13 2022] DELETE /identity/v3/projects/a5fed4f0efcc4c7184029437668e4286 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:45:14.013128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.013475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.013839 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.034327 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:14.035565 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:14.064493 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:14.094694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1353/2726] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:14.104981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.105340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.105704 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.138496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:14.139758 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:14.149828 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1374/2727] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:14.154286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.164673 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96eb0-28ed-43a9-8e73-962fdf5075c2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.170318 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.170933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.171206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.171533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.172237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:14.172518 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.174182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.186590 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1354/2728] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:45:14.190864 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.200359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a15a-5a76-4305-9d70-3fc2c2ec6a53 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.206145 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.206857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.207127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.207391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.208057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.210000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.241445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1375/2729] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:14 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:14.245887 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.256653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1c18-dfd8-46bf-a6bf-01e0b63be0da admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.311859 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.312488 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.312797 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.313032 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.313703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.315633 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.337059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1355/2730] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:14 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:14.341485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.352313 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6ca2e-4ec5-47df-84ee-8cbabbebec41 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.358031 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.358638 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.358914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.359180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.359877 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.361526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.372546 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1376/2731] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:14.377122 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.387939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9c43-2242-4127-94a1-369d1a5f8755 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.393495 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8f597-633c-47b1-be03-ded85fc110e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.394229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f597-633c-47b1-be03-ded85fc110e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.394503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f597-633c-47b1-be03-ded85fc110e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.394768 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8f597-633c-47b1-be03-ded85fc110e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.395483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8f597-633c-47b1-be03-ded85fc110e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.397145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8f597-633c-47b1-be03-ded85fc110e9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.407931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1356/2732] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:14.412321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.422560 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734109d0-a25f-4514-bc42-74757bfe6032 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.427324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.427983 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.428266 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.428530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] PATH_INFO: `/v3/projects/5dadb86207464421a56b30c6a8403f5f/users/d7b0af36f7744877bfde2fb7ce635ab0/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.443508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dadb86207464421a56b30c6a8403f5f, user_id=d7b0af36f7744877bfde2fb7ce635ab0, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.446197 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.454844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:14.461415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1377/2733] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:14 2022] PUT /identity/v3/projects/5dadb86207464421a56b30c6a8403f5f/users/d7b0af36f7744877bfde2fb7ce635ab0/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:14.471721 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.472019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.472369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.501082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] MFA Rules not processed for user `d7b0af36f7744877bfde2fb7ce635ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:14.502650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:14.566451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1357/2734] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:14.578422 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:14.580426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:14.638580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81aa1d-5664-467e-b4e4-bd98f30a9484 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:14.642851 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:14.643456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:14.643802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:14.643960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:14.652982 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:14.654394 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:14.663377 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1378/2735] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:20.616933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:20.618489 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:20.653581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:20.723180 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad8f54-0dd7-41e7-b6e0-f80f34f02a70 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:20.728823 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:20.730186 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:20.730533 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:20.730739 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:20.732954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:20.745487 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:20.747090 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93731258-9091-47b0-ba80-4cfa6e7855a5 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:20.757526 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1358/2736] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:24.659626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=27 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:28.344786 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:28.414517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201272b-1656-4eb2-8113-fc181786921f service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:28.419426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:28.420172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:28.420554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:28.421022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] PATH_INFO: `/v3/users/d7b0af36f7744877bfde2fb7ce635ab0` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:28.422228 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7b0af36f7744877bfde2fb7ce635ab0)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:28.424406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:28.520675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7b0af36f7744877bfde2fb7ce635ab0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:28.542286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7b0af36f7744877bfde2fb7ce635ab0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:28.543645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7b0af36f7744877bfde2fb7ce635ab0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:28.561128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7b0af36f7744877bfde2fb7ce635ab0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:28.563678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7b0af36f7744877bfde2fb7ce635ab0'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:28.601449 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1379/2737] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:28 2022] DELETE /identity/v3/users/d7b0af36f7744877bfde2fb7ce635ab0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:28.606002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:28.606929 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:28.607422 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:28.629169 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:28.630697 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:28.689504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1359/2738] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:28.699806 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:28.701416 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:28.783032 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f35d2-860b-42f8-8bd1-c50944da07f6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:28.787220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd84fb-896d-4274-a529-e47a4320febd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:28.787934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd84fb-896d-4274-a529-e47a4320febd service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:28.788221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd84fb-896d-4274-a529-e47a4320febd service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:28.788585 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd84fb-896d-4274-a529-e47a4320febd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:28.798210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd84fb-896d-4274-a529-e47a4320febd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:28.799641 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd84fb-896d-4274-a529-e47a4320febd service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:28.809756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1380/2739] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:45:29.095988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.097589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:29.109968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7ef92-28a0-4a63-8c65-c1bed3e51826 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.115008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.115677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.115930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.116194 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] PATH_INFO: `/v3/projects/5dadb86207464421a56b30c6a8403f5f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.117419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dadb86207464421a56b30c6a8403f5f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.119374 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.165840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dadb86207464421a56b30c6a8403f5f', 'request_id': 'req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:29.171854 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dadb86207464421a56b30c6a8403f5f', 'request_id': 'req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:29.177684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dadb86207464421a56b30c6a8403f5f', 'request_id': 'req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:29.194774 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] The token cache is being invalidate because project 5dadb86207464421a56b30c6a8403f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:29.195140 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:29.196040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eae9a54-a800-4f50-b8d5-e08ec8f987b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:29.198361 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1360/2740] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:29 2022] DELETE /identity/v3/projects/5dadb86207464421a56b30c6a8403f5f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:29.203602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.203883 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.204178 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.212025 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:29.226975 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:29.228262 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:29.292444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1381/2741] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:29.301308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.301600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.301875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.321927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:29.323117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:29.331575 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1361/2742] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:29.335890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.349203 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd1fe2-86ea-481c-bfee-aac42ffd8ef5 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.355074 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.355751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.356033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.356299 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.357030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:29.357327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.358926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.371424 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1382/2743] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:45:29.376117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.385991 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2407e0e-c2e8-4c5c-9457-938dd5f393c8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.391142 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.391745 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.392019 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.392289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.392984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.394913 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.409805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:29.423939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1362/2744] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:29 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:29.428463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.438618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14058dbe-d61e-4ecb-89c5-50835a464283 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.508457 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.509132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.509379 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.509794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.510486 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.512418 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.537759 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1383/2745] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:29 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:29.542309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.558059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5a317-2e06-457f-b515-916d317bef26 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.565762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.566380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.566706 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.567093 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.568222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.569870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.580128 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1363/2746] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:29.584066 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.594474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29df69-b4db-4df5-8cf7-34f3ca3c79cb admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.600132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6069-4b12-4a3a-b63e-9431430bb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.600665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6069-4b12-4a3a-b63e-9431430bb08c admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.600990 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6069-4b12-4a3a-b63e-9431430bb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.601258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6069-4b12-4a3a-b63e-9431430bb08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.601948 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a6069-4b12-4a3a-b63e-9431430bb08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.603527 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a6069-4b12-4a3a-b63e-9431430bb08c admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.614038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1384/2747] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:29.618234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.629558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85455e-2f58-41b3-8de9-8241ce50dbee admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.635072 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.635755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.636033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.636307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] PATH_INFO: `/v3/projects/897eef74408f4973bfb541b7f92365c7/users/08ee7fc8c18641b2ac4c1400638d54f1/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.653217 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897eef74408f4973bfb541b7f92365c7, user_id=08ee7fc8c18641b2ac4c1400638d54f1, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.656126 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.688269 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1364/2748] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:29 2022] PUT /identity/v3/projects/897eef74408f4973bfb541b7f92365c7/users/08ee7fc8c18641b2ac4c1400638d54f1/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:29.688375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.688375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.688375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.721108 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:29.722526 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:29.810537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1385/2749] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:29.824001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:29.824926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:29.893321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208b769-1e13-43c6-a33d-85d619bd7f10 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:29.897619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc510a-98f1-4ba0-97e5-d5f52b533e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:29.898236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc510a-98f1-4ba0-97e5-d5f52b533e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:29.898507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc510a-98f1-4ba0-97e5-d5f52b533e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:29.898769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc510a-98f1-4ba0-97e5-d5f52b533e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:29.909022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc510a-98f1-4ba0-97e5-d5f52b533e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:29.910532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc510a-98f1-4ba0-97e5-d5f52b533e50 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:29.929277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1365/2750] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:32.449399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:32.450884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:32.471893 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fbca94-f97b-47ec-94e8-d75373193c0a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:32.486636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:32.487314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:32.487581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:32.487855 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] PATH_INFO: `/v3/users/bcb2733aec624f72badfa4a5384ab295` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:32.489542 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb2733aec624f72badfa4a5384ab295)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:32.491569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:32.502564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:32.571198 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb2733aec624f72badfa4a5384ab295'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:32.582106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb2733aec624f72badfa4a5384ab295'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:32.582934 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb2733aec624f72badfa4a5384ab295'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:32.593292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb2733aec624f72badfa4a5384ab295'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:32.594308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb2733aec624f72badfa4a5384ab295'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:32.628288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1386/2751] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:32 2022] DELETE /identity/v3/users/bcb2733aec624f72badfa4a5384ab295 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:32.632351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:32.632849 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:32.632963 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:32.675239 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:32.677046 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:32.825146 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1366/2752] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:32.841247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:32.843259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:32.973671 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dedfc-cb4d-4b06-a73d-cbb47b57fbfc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:32.977976 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c58f5d-8beb-4c96-8f04-56fab2e476d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:32.978695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58f5d-8beb-4c96-8f04-56fab2e476d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:32.978972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58f5d-8beb-4c96-8f04-56fab2e476d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:32.979391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58f5d-8beb-4c96-8f04-56fab2e476d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:33.002219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c58f5d-8beb-4c96-8f04-56fab2e476d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:33.003832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c58f5d-8beb-4c96-8f04-56fab2e476d7 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:33.031490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1387/2753] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:45:33.832726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:33.834707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:33.851509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395d43-4a08-4f18-9d0f-e4925af2f98f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:33.862983 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:33.863735 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:33.864015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:33.864289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] PATH_INFO: `/v3/projects/4f0001bd61ec499395c7989d7eb804f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:33.865635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0001bd61ec499395c7989d7eb804f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:33.867800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:33.937769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0001bd61ec499395c7989d7eb804f0', 'request_id': 'req-93324d08-fedd-4996-922c-28062c4cb619'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:33.947413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0001bd61ec499395c7989d7eb804f0', 'request_id': 'req-93324d08-fedd-4996-922c-28062c4cb619'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:33.950161 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:33.961056 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0001bd61ec499395c7989d7eb804f0', 'request_id': 'req-93324d08-fedd-4996-922c-28062c4cb619'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:34.000244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] The token cache is being invalidate because project 4f0001bd61ec499395c7989d7eb804f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:34.000545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:34.001484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-93324d08-fedd-4996-922c-28062c4cb619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:34.003406 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1367/2754] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:33 2022] DELETE /identity/v3/projects/4f0001bd61ec499395c7989d7eb804f0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:34.008548 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.009063 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.009410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.035929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:34.037713 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:34.174857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1388/2755] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:34.182803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.182803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.183112 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.220349 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:34.221597 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:34.256916 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1368/2756] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:34.258091 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:34.272570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462c59e-66dd-4b4d-9f41-b67d933263e9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:34.278794 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:34.279576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.279953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.280320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.281171 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:34.281910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:34.284614 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:34.301861 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1389/2757] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:45:34.307063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:34.319337 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5236ee-c4a6-4c3f-b567-ff9ca95f04fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:34.326416 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:34.327275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.327741 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.328214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.329805 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:34.332121 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:34.383888 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1369/2758] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:34 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:34.384539 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:34.400891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbac230-cdd6-42ae-b3df-d40a5d70e668 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:34.414183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:34.502734 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:34.503760 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.504041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.505925 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.506763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:34.510839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:34.560329 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1390/2759] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:34 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:34.564248 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:34.588474 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6d8d5-798b-43ea-83b0-2607f6e05156 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:34.598100 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:34.598746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.599101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.599366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.600074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:34.601938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:34.621655 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1370/2760] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:34.623172 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:34.639384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ee0c7-8a76-4cc5-9bca-0913a2b9dcf4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:34.651083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5241ac8-58f7-4435-a675-9e57900a222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:34.651916 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5241ac8-58f7-4435-a675-9e57900a222a admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.652295 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5241ac8-58f7-4435-a675-9e57900a222a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.653220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5241ac8-58f7-4435-a675-9e57900a222a admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.654043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5241ac8-58f7-4435-a675-9e57900a222a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:34.657994 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5241ac8-58f7-4435-a675-9e57900a222a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:34.695937 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1391/2761] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:34.701314 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:34.723255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf633fb9-f01f-4b62-871e-72cf56aa5b6e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:34.730322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:34.731037 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.731317 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.731593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] PATH_INFO: `/v3/projects/c77cf0c8588c4498b4ddbd50b0e87dcc/users/6a14bfe0937448bbbf58f12264264efd/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.789704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77cf0c8588c4498b4ddbd50b0e87dcc, user_id=6a14bfe0937448bbbf58f12264264efd, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:34.794244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:34.825891 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1371/2762] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:34 2022] PUT /identity/v3/projects/c77cf0c8588c4498b4ddbd50b0e87dcc/users/6a14bfe0937448bbbf58f12264264efd/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:34.836206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:34.836781 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:34.837210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:34.875924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] MFA Rules not processed for user `6a14bfe0937448bbbf58f12264264efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:34.877622 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:34.983206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1392/2763] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:34.987616 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.057196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd9456-a80d-4e7f-9bfe-4320d410cf79 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.062221 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c4e67-692f-444b-a9a8-c6d87754a7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.062874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c4e67-692f-444b-a9a8-c6d87754a7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.063125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c4e67-692f-444b-a9a8-c6d87754a7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.063394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c4e67-692f-444b-a9a8-c6d87754a7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.064316 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c4e67-692f-444b-a9a8-c6d87754a7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.066000 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c4e67-692f-444b-a9a8-c6d87754a7e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.105580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1372/2764] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:34 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:35.120435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.134314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9259f38-f3ff-4dd5-9602-b51b8332f9f9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.184628 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.184953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.185229 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.210039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.210770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.211041 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.211308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.211969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.214647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.218520 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] MFA Rules not processed for user `08ee7fc8c18641b2ac4c1400638d54f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:35.219911 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:35.266300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1393/2765] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:35.267931 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.285827 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03caf4-935a-4900-a2ca-22714cfee050 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.291995 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.293000 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.293571 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.293986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.294969 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.296743 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.301072 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:35.307261 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:35.321115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1394/2766] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:35.322450 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.326340 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1373/2767] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:35.329864 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.332577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:35.335506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13effdb7-383b-44ad-85e4-20a398e85e62 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.341665 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.342378 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.342708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.343155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.343871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.346501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.359230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c596e4b-a500-4ac8-8137-03a865f1d269 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.361468 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1395/2768] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:35.365972 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.366150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.367017 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.367427 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.367764 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.368511 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.370557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.379735 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba465c-42fc-4a7c-b112-c8e998a249ec admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.386222 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85401a77-58fd-4938-9d35-529fae1abee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.387148 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85401a77-58fd-4938-9d35-529fae1abee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.387602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85401a77-58fd-4938-9d35-529fae1abee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.388027 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85401a77-58fd-4938-9d35-529fae1abee6 admin admin] PATH_INFO: `/v3/projects/9bb710e2c07e473095ab4c59f8454a84/users/e4d63beadba649dd9d50ceedc4ffe969/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.408714 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1374/2769] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:35.413740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.415385 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85401a77-58fd-4938-9d35-529fae1abee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb710e2c07e473095ab4c59f8454a84, user_id=e4d63beadba649dd9d50ceedc4ffe969, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.418098 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85401a77-58fd-4938-9d35-529fae1abee6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.453208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1396/2770] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:35 2022] PUT /identity/v3/projects/9bb710e2c07e473095ab4c59f8454a84/users/e4d63beadba649dd9d50ceedc4ffe969/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:35.465581 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.466242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.466723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.502481 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] MFA Rules not processed for user `e4d63beadba649dd9d50ceedc4ffe969`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:35.505411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:35.536717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e07dc-9805-46d0-a981-9e2ba8328010 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.595927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1397/2771] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:35.600099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.604445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.605135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.605403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.605671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.606332 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.608255 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.610956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088c7ed-13cb-44e9-94ef-d614e9984a1b None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.615442 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.616033 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.616304 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.616580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.617300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.619216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.632986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1375/2772] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:35.636677 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.647834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc595d-efea-4323-8c01-5f4fed823668 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.652579 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.653241 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.653375 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1398/2773] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:35.653582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.653780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.654613 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.656283 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.657580 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.670053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1376/2774] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:45:35.670977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.681145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ef46f-09e8-4767-8ea0-6d93702cbd45 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.728667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0ebe-b249-40ca-8ca3-32104faf3044 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.734723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.735490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.735900 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.736343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.737152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.739250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.752233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.753013 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.753351 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.753621 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] PATH_INFO: `/v3/projects/944889ed3b654f6e8c390f9fe58b1855/users/9b8544f860e24a838e7f417f5f4df3ff/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.756324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:35.767132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1399/2775] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:35 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:35.771601 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.772792 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] RBAC: Authorizing `identity:create_grant(project_id=944889ed3b654f6e8c390f9fe58b1855, user_id=9b8544f860e24a838e7f417f5f4df3ff, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.777324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.782307 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a7d2-59ca-469a-bf05-405a7ef68139 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.787515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.788747 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.789061 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.789423 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.790135 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.792065 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.798767 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1377/2776] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:35 2022] PUT /identity/v3/projects/944889ed3b654f6e8c390f9fe58b1855/users/9b8544f860e24a838e7f417f5f4df3ff/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:35.802632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.805007 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1400/2777] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:35.807509 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.821964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729908e9-63a3-4978-a498-4b80633dacc9 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.832939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:35.897814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d31dd6-d120-4640-9e27-2aa74340455b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.906524 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.908956 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.909226 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.909567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.910567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.913675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.929758 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1378/2778] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:45:35.930126 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.931209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.931209 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.931425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:35.931722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] PATH_INFO: `/v3/projects/d78d813dca9c46c5911d628f64c0ba21/users/2dbaa02f289040dcaedfa3f0a8b95e6a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.945667 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919eb5b-b64b-4219-bb02-ccbc1c24d24b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:35.951716 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:35.952604 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:35.953256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:35.953682 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] PATH_INFO: `/v3/projects/944889ed3b654f6e8c390f9fe58b1855/users/9b8544f860e24a838e7f417f5f4df3ff/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:35.956040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78d813dca9c46c5911d628f64c0ba21, user_id=2dbaa02f289040dcaedfa3f0a8b95e6a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.956563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=944889ed3b654f6e8c390f9fe58b1855, user_id=9b8544f860e24a838e7f417f5f4df3ff, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:35.959475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.960853 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:35.978453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1379/2779] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:35 2022] PUT /identity/v3/projects/944889ed3b654f6e8c390f9fe58b1855/users/9b8544f860e24a838e7f417f5f4df3ff/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:35.987473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1401/2780] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:35 2022] PUT /identity/v3/projects/d78d813dca9c46c5911d628f64c0ba21/users/2dbaa02f289040dcaedfa3f0a8b95e6a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:45:35.994206 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:36.073384 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27bf4-18f1-438a-9328-90382e443889 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:36.164130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aca61-2924-4c55-b858-70cf3f4a435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:36.165263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aca61-2924-4c55-b858-70cf3f4a435f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:36.165263 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aca61-2924-4c55-b858-70cf3f4a435f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:36.165540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aca61-2924-4c55-b858-70cf3f4a435f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:36.166354 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aca61-2924-4c55-b858-70cf3f4a435f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:36.167995 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aca61-2924-4c55-b858-70cf3f4a435f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:36.180611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1380/2781] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:36.184809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:36.209200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74625e2d-913d-4a1a-8b12-fae1493d3f5a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:36.217648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:36.218479 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:36.219370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:36.219752 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] PATH_INFO: `/v3/projects/d78d813dca9c46c5911d628f64c0ba21/users/2dbaa02f289040dcaedfa3f0a8b95e6a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:36.221929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78d813dca9c46c5911d628f64c0ba21, user_id=2dbaa02f289040dcaedfa3f0a8b95e6a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:36.224808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:36.237788 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1402/2782] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:36 2022] PUT /identity/v3/projects/d78d813dca9c46c5911d628f64c0ba21/users/2dbaa02f289040dcaedfa3f0a8b95e6a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:45:36.248930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5c73e-0542-4301-8dd6-045fb9f77190 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:36.249288 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5c73e-0542-4301-8dd6-045fb9f77190 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:36.249763 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5c73e-0542-4301-8dd6-045fb9f77190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:36.270390 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:36.281825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0ab5c73e-0542-4301-8dd6-045fb9f77190 None None] MFA Rules not processed for user `2dbaa02f289040dcaedfa3f0a8b95e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:36.283455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0ab5c73e-0542-4301-8dd6-045fb9f77190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:36.377440 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1381/2783] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:36.389146 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:36.391547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:36.495382 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91369fc6-a1db-4de8-8f4b-2262a6371ba0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:36.499141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:36.499836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:36.500187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:36.500529 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:36.567413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:36.568956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:36.584028 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1403/2784] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:36.812593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:36.812909 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:36.813183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:36.854370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:36.857366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:37.011748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1382/2785] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:45:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:37.209431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:37.211997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:37.237319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:37.323652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4ae1-68bd-442e-b6d5-87051939122e service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:37.330651 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a61ca-6df8-4a66-8220-33af252b815a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:37.331372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a61ca-6df8-4a66-8220-33af252b815a service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:37.331723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a61ca-6df8-4a66-8220-33af252b815a service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:37.332101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a61ca-6df8-4a66-8220-33af252b815a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:37.349404 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a61ca-6df8-4a66-8220-33af252b815a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:37.351043 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a61ca-6df8-4a66-8220-33af252b815a service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:37.362095 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1404/2786] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:45:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:45:37.878182 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:37.882280 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:38.024258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66622e21-09bc-45f5-80ee-1775fcf9a647 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:38.032196 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:38.032924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:38.033260 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:38.033593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] PATH_INFO: `/v3/users/f5d3517f51b74230964f1e2b29371e0b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:38.037212 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d3517f51b74230964f1e2b29371e0b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:38.039740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:38.092580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:38.121195 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d3517f51b74230964f1e2b29371e0b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.134938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d3517f51b74230964f1e2b29371e0b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.136652 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d3517f51b74230964f1e2b29371e0b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.154694 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d3517f51b74230964f1e2b29371e0b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.165650 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d3517f51b74230964f1e2b29371e0b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.182510 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1383/2787] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:37 2022] DELETE /identity/v3/users/f5d3517f51b74230964f1e2b29371e0b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:38.243738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:38.244629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:38.245106 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:38.293763 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:38.296807 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:38.324262 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:38.330114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:38.466748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1405/2788] 10.222.0.49 () {58 vars in 967 bytes} [Sun Jul 31 18:45:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:38.476547 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e49e-a273-4161-9156-49d21428c832 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:38.484999 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:38.486490 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:38.495834 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:38.496490 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:38.496777 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:38.499251 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:38.510170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:38.650198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:38.652592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:38.659882 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001516ac-8812-444a-9ec4-d0c24f9eef0e None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:38.684839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832d5c-1111-4ce6-9758-423dccebaec0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:38.685811 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832d5c-1111-4ce6-9758-423dccebaec0 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:38.686092 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832d5c-1111-4ce6-9758-423dccebaec0 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:38.686363 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832d5c-1111-4ce6-9758-423dccebaec0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:38.697452 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1384/2789] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:38.700036 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:38.701519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:38.709275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832d5c-1111-4ce6-9758-423dccebaec0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:38.711572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832d5c-1111-4ce6-9758-423dccebaec0 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:38.722686 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1406/2790] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:45:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:45:38.821165 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:38.835475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363924b-c403-4cd3-a2ed-890b13403a99 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:38.850620 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:38.851464 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:38.851986 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:38.852355 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] PATH_INFO: `/v3/projects/9540ab0260cc4c0ca76036b79bb5f506` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:38.854247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9540ab0260cc4c0ca76036b79bb5f506)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:38.856301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:38.950008 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9540ab0260cc4c0ca76036b79bb5f506', 'request_id': 'req-38f658f3-edca-460e-ad59-b53a5e0c6202'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.962418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9540ab0260cc4c0ca76036b79bb5f506', 'request_id': 'req-38f658f3-edca-460e-ad59-b53a5e0c6202'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.969150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9540ab0260cc4c0ca76036b79bb5f506', 'request_id': 'req-38f658f3-edca-460e-ad59-b53a5e0c6202'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.993648 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] The token cache is being invalidate because project 9540ab0260cc4c0ca76036b79bb5f506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:38.993941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.995101 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-38f658f3-edca-460e-ad59-b53a5e0c6202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:38.997099 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1385/2791] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:38 2022] DELETE /identity/v3/projects/9540ab0260cc4c0ca76036b79bb5f506 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:39.002710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791821-166e-4463-bbb6-8a688b846a5d None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.003237 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791821-166e-4463-bbb6-8a688b846a5d None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.003808 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791821-166e-4463-bbb6-8a688b846a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.047583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-0c791821-166e-4463-bbb6-8a688b846a5d None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:39.048857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0c791821-166e-4463-bbb6-8a688b846a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:39.099607 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.101198 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.101726 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.115831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:39.138454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:39.141309 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:39.159308 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1407/2792] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:39.168611 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.168938 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.169202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.207431 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:39.208733 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:39.234077 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1408/2793] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:39.238627 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.248632 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1386/2794] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:45:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:45:39.252267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b904aa1-fefb-439d-92fd-41473b152a7a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.260254 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.261097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.261444 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.261776 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.270895 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:39.271143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.271522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.273986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.278039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:39.297783 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1409/2795] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 18:45:39.301862 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.320910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042dc1f-400c-4dcc-9ec4-ed1bd66597f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.329559 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.330323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.330606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.331057 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.331756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.334886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.381115 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1410/2796] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:39 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:39.385140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.446178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4875d31-8d31-4d16-b0e7-93667e7fc273 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.539573 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:39.549323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f90eb-4c30-4bcf-b06b-46b4199793e0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.561832 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.563247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.563534 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.563823 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.566037 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.576283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.601002 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.601815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.602095 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.602443 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.636569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1411/2797] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:39 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:39.641224 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.661366 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81643994-29f9-43a3-89ce-68257fa15eb6 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.689816 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.690629 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.690914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.691496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.692210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.694252 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.695219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.696657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.708408 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1387/2798] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:45:39.726183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1412/2799] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:45:39.728284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.728927 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.731047 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:39.742455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc07d9-74f4-4e09-8f75-3caed923039f service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.773058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.773676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.774563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.774563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.776425 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.777307 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.793370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1388/2800] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:39.801243 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:39.825128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b20a1-5317-4cda-b5d6-f0d0163a8b83 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.826214 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339dcabc-d1c7-443e-85d8-2fbf1f042013 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:39.829789 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.830435 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.830868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.831140 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.833399 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:39.846070 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8a0a8-9945-4dd3-9484-08e5da77cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:39.853080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8a0a8-9945-4dd3-9484-08e5da77cbee admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.853430 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8a0a8-9945-4dd3-9484-08e5da77cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.853898 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8a0a8-9945-4dd3-9484-08e5da77cbee admin admin] PATH_INFO: `/v3/projects/11018257977c4b8ba8f008c0c6ae65fd/users/64d0451d06294c968712f411f90f224e/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.875723 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:39.884599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8a0a8-9945-4dd3-9484-08e5da77cbee admin admin] RBAC: Authorizing `identity:create_grant(project_id=11018257977c4b8ba8f008c0c6ae65fd, user_id=64d0451d06294c968712f411f90f224e, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.889234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8a0a8-9945-4dd3-9484-08e5da77cbee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.922695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1389/2801] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:39 2022] PUT /identity/v3/projects/11018257977c4b8ba8f008c0c6ae65fd/users/64d0451d06294c968712f411f90f224e/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:39.939231 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:39.941551 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688cb68-657a-4631-9e3c-71506d3bb47a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:39.941894 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688cb68-657a-4631-9e3c-71506d3bb47a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:39.942172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688cb68-657a-4631-9e3c-71506d3bb47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:39.942748 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:39.977438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1413/2802] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:40.034766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b688cb68-657a-4631-9e3c-71506d3bb47a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:40.039006 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b688cb68-657a-4631-9e3c-71506d3bb47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:40.230855 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1390/2803] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:40.238141 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:40.239697 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:40.379104 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e3499-b45e-4cee-bbc5-e718b631d9a8 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:40.458325 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b83b05-8126-4c51-af98-9099bbe2bc0a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:40.458638 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b83b05-8126-4c51-af98-9099bbe2bc0a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:40.458883 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b83b05-8126-4c51-af98-9099bbe2bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:40.459890 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:40.460572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:40.460898 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:40.461174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:40.485826 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:40.488958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:40.495610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c7b83b05-8126-4c51-af98-9099bbe2bc0a None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:40.496863 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c7b83b05-8126-4c51-af98-9099bbe2bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:40.515113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1414/2804] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:40.559921 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:40.588847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1391/2805] 10.222.0.49 () {58 vars in 967 bytes} [Sun Jul 31 18:45:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:40.603301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:40.720384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26498eb3-c0cd-48b6-87d5-e54f6481e7d2 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:40.733525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:40.734330 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:40.734618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:40.734887 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:40.757647 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:40.759489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:40.779515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1415/2806] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:45:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:45:43.425412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:43.426417 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:43.426994 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:43.498009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] MFA Rules not processed for user `9b8544f860e24a838e7f417f5f4df3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:43.499614 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:43.580182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:43.581680 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:43.604193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d2790-4602-4f87-aaa6-f7dc8ff54eb6 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:43.610180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:43.610854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:43.611132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:43.611406 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] PATH_INFO: `/v3/users/c7dd082069b3423a8686cd809c8d4110` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:43.612709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7dd082069b3423a8686cd809c8d4110)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:43.614849 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:43.622105 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:43.626973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1392/2807] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:43.641435 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:43.643136 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:43.680711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7dd082069b3423a8686cd809c8d4110'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:43.693189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7dd082069b3423a8686cd809c8d4110'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:43.694311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7dd082069b3423a8686cd809c8d4110'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:43.708400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7dd082069b3423a8686cd809c8d4110'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:43.709206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-048f93c5-7799-4ffa-967c-4d054972003a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7dd082069b3423a8686cd809c8d4110'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:43.734357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1416/2808] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:43 2022] DELETE /identity/v3/users/c7dd082069b3423a8686cd809c8d4110 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:43.740894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:43.741557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:43.741844 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:43.748981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beff91-cfec-4c45-95e9-41c845fd28c4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:43.765180 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:43.766699 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:43.814234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e84fa2255614cde84495eedcc0971a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:43.814895 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:43.815178 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:43.815448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:43.817988 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:43.837155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1417/2809] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:43.851438 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:43.852986 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:43.875468 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:43.900939 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:43.903581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493ef1e-f55c-46b7-9438-af73ec44a271 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:43.914237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1393/2810] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:45:43.925215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb916f-2a4a-4968-8848-6734bf76fc4a None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:43.929327 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:43.930024 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:43.930372 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:43.930773 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:43.946836 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:43.948524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:43.960744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1418/2811] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:45:44.116903 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:44.117189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:44.117457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:44.162594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:44.165611 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:44.185681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:44.189569 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:44.206440 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0fb28-7ac1-4129-b92f-7edfd929542b service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:44.240214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:44.241210 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:44.241695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:44.241978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] PATH_INFO: `/v3/projects/6a503222f5964076a75d291a6530097c` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:44.243185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a503222f5964076a75d291a6530097c)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:44.245551 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:44.267045 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1394/2812] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:45:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:44.287087 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:44.289825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:44.305512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:44.352458 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a503222f5964076a75d291a6530097c', 'request_id': 'req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.367446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a503222f5964076a75d291a6530097c', 'request_id': 'req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.376276 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a503222f5964076a75d291a6530097c', 'request_id': 'req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.397445 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] The token cache is being invalidate because project 6a503222f5964076a75d291a6530097c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:44.398019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.398177 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8457628-34df-4ae9-adce-38a97afec1a9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:44.399069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.401537 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1419/2813] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:44 2022] DELETE /identity/v3/projects/6a503222f5964076a75d291a6530097c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:45:44.477867 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:44.478532 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:44.478815 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:44.479234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:44.481974 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:44.531765 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:44.572500 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:44.574339 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09637cdd-f442-4efe-9487-77b8d536bda3 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:44.584674 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1395/2814] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:45:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:45:44.742250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:44.744675 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:44.854830 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ecd83-c639-47ac-b5da-20d51bdc9c93 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:44.860714 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:44.861426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:44.861737 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:44.862015 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] PATH_INFO: `/v3/users/6a14bfe0937448bbbf58f12264264efd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:44.864013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a14bfe0937448bbbf58f12264264efd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:44.866128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:44.958340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a14bfe0937448bbbf58f12264264efd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.974567 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a14bfe0937448bbbf58f12264264efd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.975250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a14bfe0937448bbbf58f12264264efd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.995249 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a14bfe0937448bbbf58f12264264efd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:44.996136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a14bfe0937448bbbf58f12264264efd'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:45.034300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1420/2815] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:44 2022] DELETE /identity/v3/users/6a14bfe0937448bbbf58f12264264efd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:45.039445 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:45.039717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:45.039984 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:45.079938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:45.081204 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:45.225413 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1396/2816] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:45.243413 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:45.244908 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:45.323485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:45.380493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea32f2f-a88a-4c5b-a3b7-95114d553aea admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:45.394238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:45.395174 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:45.395637 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:45.396013 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:45.414326 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:45.416544 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:45.458206 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1421/2817] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:45.968009 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:45.969504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:45.996540 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51884ff4-6426-4eed-af32-73e2263cd838 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:46.020589 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:46.021541 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:46.021981 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:46.022403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] PATH_INFO: `/v3/projects/c77cf0c8588c4498b4ddbd50b0e87dcc` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:46.031364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77cf0c8588c4498b4ddbd50b0e87dcc)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:46.033680 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:46.143528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77cf0c8588c4498b4ddbd50b0e87dcc', 'request_id': 'req-beb5febc-0758-48f1-b668-f0d5c298b824'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.156567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77cf0c8588c4498b4ddbd50b0e87dcc', 'request_id': 'req-beb5febc-0758-48f1-b668-f0d5c298b824'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.167545 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77cf0c8588c4498b4ddbd50b0e87dcc', 'request_id': 'req-beb5febc-0758-48f1-b668-f0d5c298b824'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.195184 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] The token cache is being invalidate because project c77cf0c8588c4498b4ddbd50b0e87dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:46.195470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.196228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.198392 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1397/2818] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:45 2022] DELETE /identity/v3/projects/c77cf0c8588c4498b4ddbd50b0e87dcc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:46.202824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:46.217954 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cc13a-adf0-4d8e-b25d-697e8f4ca82a service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:46.348516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:46.349259 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:46.349530 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:46.349802 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] PATH_INFO: `/v3/users/e4d63beadba649dd9d50ceedc4ffe969` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:46.350984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d63beadba649dd9d50ceedc4ffe969)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:46.354123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:46.467775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d63beadba649dd9d50ceedc4ffe969'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.496427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d63beadba649dd9d50ceedc4ffe969'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.497107 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d63beadba649dd9d50ceedc4ffe969'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.526755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d63beadba649dd9d50ceedc4ffe969'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.534398 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d63beadba649dd9d50ceedc4ffe969'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:46.585185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1422/2819] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:46 2022] DELETE /identity/v3/users/e4d63beadba649dd9d50ceedc4ffe969 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:47.120469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:47.123282 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:47.162201 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5febc-0758-48f1-b668-f0d5c298b824 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:47.288848 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:47.308207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:47.309364 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:47.311796 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:47.312172 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] PATH_INFO: `/v3/projects/9bb710e2c07e473095ab4c59f8454a84` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:47.313869 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb710e2c07e473095ab4c59f8454a84)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:47.319666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:47.445144 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb710e2c07e473095ab4c59f8454a84', 'request_id': 'req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.462718 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb710e2c07e473095ab4c59f8454a84', 'request_id': 'req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.495759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb710e2c07e473095ab4c59f8454a84', 'request_id': 'req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.535581 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] The token cache is being invalidate because project 9bb710e2c07e473095ab4c59f8454a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:47.535931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.536625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-03f5e769-6cd6-4fc0-8016-7e7479e4de6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.541156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1398/2820] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:47 2022] DELETE /identity/v3/projects/9bb710e2c07e473095ab4c59f8454a84 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:47.542669 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:47.544401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:47.575283 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb85047-e9eb-41f4-b46f-7a5224413d5b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:47.666678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:47.667236 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:47.667640 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:47.669116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:47.689965 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:47.690825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:47.691359 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:47.691645 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] PATH_INFO: `/v3/users/2dbaa02f289040dcaedfa3f0a8b95e6a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:47.693067 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dbaa02f289040dcaedfa3f0a8b95e6a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:47.695050 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:47.735297 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] MFA Rules not processed for user `64d0451d06294c968712f411f90f224e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:47.736622 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:47.792899 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dbaa02f289040dcaedfa3f0a8b95e6a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.850054 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dbaa02f289040dcaedfa3f0a8b95e6a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.851076 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dbaa02f289040dcaedfa3f0a8b95e6a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.884973 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dbaa02f289040dcaedfa3f0a8b95e6a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.886977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dbaa02f289040dcaedfa3f0a8b95e6a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:47.958645 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1399/2821] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:47.963258 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:47.965709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:47.972572 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1423/2822] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:45:47 2022] DELETE /identity/v3/users/2dbaa02f289040dcaedfa3f0a8b95e6a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:47.997459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad4f18-1249-4cd9-b088-831fb715b3dc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.171672 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.173781 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.175492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.175955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.177755 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.183712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.233563 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:48.250933 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.252514 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1400/2823] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:47 2022] POST /identity/v3/projects => generated 377 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:45:48.268247 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f4362-14b9-4635-b34b-02269d62ef3a admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.407525 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.408332 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.409272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.409272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.410030 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.412292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.455681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1424/2824] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:48 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:48.456924 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.476688 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf587a-0261-4a4b-8460-93bb96940d6b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.493740 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.494858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.495360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.502084 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.503328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.505256 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.516512 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.518663 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:48.519860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1401/2825] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 18:45:48.523926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.548200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b633c-4717-4773-9222-65ab7d010351 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.554602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.555924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.556298 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.556667 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] PATH_INFO: `/v3/projects/d78d813dca9c46c5911d628f64c0ba21` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.558002 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78d813dca9c46c5911d628f64c0ba21)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.561068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.568536 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589e32-8671-4957-a8b2-d08727147f9b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.585461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.586426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.586558 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.586803 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] PATH_INFO: `/v3/projects/c403eb8800ad464d9fc2c7abf461699a/users/df9dfec58ce844659fd0d79d684c0d2d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.612666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c403eb8800ad464d9fc2c7abf461699a, user_id=df9dfec58ce844659fd0d79d684c0d2d, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.620977 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.632894 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78d813dca9c46c5911d628f64c0ba21', 'request_id': 'req-66905125-90ee-4725-8835-6e4c0f252507'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:48.642389 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1402/2826] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:48 2022] PUT /identity/v3/projects/c403eb8800ad464d9fc2c7abf461699a/users/df9dfec58ce844659fd0d79d684c0d2d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:48.643150 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78d813dca9c46c5911d628f64c0ba21', 'request_id': 'req-66905125-90ee-4725-8835-6e4c0f252507'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:48.646886 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:DELETE=15,keystone:INSERT=4 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:48.647979 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.652492 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78d813dca9c46c5911d628f64c0ba21', 'request_id': 'req-66905125-90ee-4725-8835-6e4c0f252507'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:48.670820 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] The token cache is being invalidate because project d78d813dca9c46c5911d628f64c0ba21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:45:48.671274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:48.672046 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-66905125-90ee-4725-8835-6e4c0f252507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:45:48.674133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1425/2827] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:45:48 2022] DELETE /identity/v3/projects/d78d813dca9c46c5911d628f64c0ba21 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:45:48.679160 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55402301-15ca-4f37-bcae-40f24e6dd097 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.679441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55402301-15ca-4f37-bcae-40f24e6dd097 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.679809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55402301-15ca-4f37-bcae-40f24e6dd097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.700073 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-55402301-15ca-4f37-bcae-40f24e6dd097 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:48.701483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-55402301-15ca-4f37-bcae-40f24e6dd097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:48.741838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae4532-fff6-4a1e-a4f3-59f94f6dd7c7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.748277 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.749491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.751289 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.752094 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.753029 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.755455 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.770235 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1403/2828] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:48.774028 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.776376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:48.779709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1426/2829] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:48.796323 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.796703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427f74-ad3e-4b14-b691-53b8f2f33fdf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.797019 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.797391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.803653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.804619 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.805077 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.805503 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] PATH_INFO: `/v3/projects/c403eb8800ad464d9fc2c7abf461699a/users/df9dfec58ce844659fd0d79d684c0d2d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.808366 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c403eb8800ad464d9fc2c7abf461699a, user_id=df9dfec58ce844659fd0d79d684c0d2d, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.812025 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.821652 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:48.823270 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:48.825341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1404/2830] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:48 2022] PUT /identity/v3/projects/c403eb8800ad464d9fc2c7abf461699a/users/df9dfec58ce844659fd0d79d684c0d2d/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:48.914321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1427/2831] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:48.918321 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.931508 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f05ba0-a26a-4609-bab8-7775c8deba9b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.937418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.938135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.938368 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.938593 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.939412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:45:48.939717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.941357 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:48.959881 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1405/2832] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:45:48.965485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:48.977494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891bd17-4a2d-4454-a69e-655172b73df9 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:48.982300 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:48.982928 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:48.983338 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:48.983616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:48.984412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:48.986494 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:49.036026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1428/2833] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:45:48 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:49.041053 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:49.054692 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447bb0-79f7-4bc3-802a-ec6b1b27d43e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:49.129711 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:49.130419 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:49.130695 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:49.130965 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:49.131634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:49.133778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:49.173034 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:49.173990 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1406/2834] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:45:49 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:45:49.194225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686a0f-ae68-4f8d-99e5-0e14d1eb5964 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:49.199219 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:49.199859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:49.200187 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:49.200462 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:49.201221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:49.203309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:49.221250 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1429/2835] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:49.226470 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:49.241441 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db508c79-34e6-4bb8-8d2c-af99c98f1725 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:49.251982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564eefce-0ccd-48c8-a2e9-ab479a1e6793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:49.252942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564eefce-0ccd-48c8-a2e9-ab479a1e6793 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:49.253499 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564eefce-0ccd-48c8-a2e9-ab479a1e6793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:49.253968 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564eefce-0ccd-48c8-a2e9-ab479a1e6793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:49.254934 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564eefce-0ccd-48c8-a2e9-ab479a1e6793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:49.257303 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564eefce-0ccd-48c8-a2e9-ab479a1e6793 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:49.278308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1407/2836] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:45:49.280173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:49.300136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c96cb-5962-412b-8ca9-2824bab459f5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:49.307978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:49.308848 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:49.309710 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:49.310275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] PATH_INFO: `/v3/projects/e95b82558df841cab6e144f21099ea6f/users/4adfedbe1b074c1b96f10c3a9d879ab1/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:49.337446 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95b82558df841cab6e144f21099ea6f, user_id=4adfedbe1b074c1b96f10c3a9d879ab1, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:49.352214 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:49.388888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1430/2837] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:45:49 2022] PUT /identity/v3/projects/e95b82558df841cab6e144f21099ea6f/users/4adfedbe1b074c1b96f10c3a9d879ab1/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:45:49.396410 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25e89a-fc3d-439d-a074-2373a3c74938 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:49.396858 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25e89a-fc3d-439d-a074-2373a3c74938 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:49.397230 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25e89a-fc3d-439d-a074-2373a3c74938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:49.460629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:49.466059 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4a25e89a-fc3d-439d-a074-2373a3c74938 None None] MFA Rules not processed for user `4adfedbe1b074c1b96f10c3a9d879ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:49.468168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4a25e89a-fc3d-439d-a074-2373a3c74938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:49.649145 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1408/2838] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:49.658751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:49.660677 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:49.725591 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:45:49.806839 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378e779-7279-41e9-bbeb-eaca222b8d58 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:49.817950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:49.819183 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:49.819531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:49.819868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:49.831978 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:49.833742 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:49.853161 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1431/2839] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:45:56.592811 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0145-f54b-42bd-a1b9-f5b4069db1de None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:56.593324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0145-f54b-42bd-a1b9-f5b4069db1de None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:56.593521 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0145-f54b-42bd-a1b9-f5b4069db1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:56.630845 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d32a0145-f54b-42bd-a1b9-f5b4069db1de None None] MFA Rules not processed for user `df9dfec58ce844659fd0d79d684c0d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:45:56.632076 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d32a0145-f54b-42bd-a1b9-f5b4069db1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:56.755476 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1409/2840] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:45:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:45:56.767709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:45:56.770655 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:57.005289 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda7e9d-fab4-4eff-89d3-260b8bbbd1e7 service cinder] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:45:57.013505 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:45:57.014256 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:45:57.014692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:45:57.015082 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:45:57.017349 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:45:57.137044 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:45:57.138784 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:45:57.152953 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1432/2841] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:45:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:46:00.794982 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ae113-7cdf-4a0f-bdb7-a4ce053cdb6a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:00.795882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ae113-7cdf-4a0f-bdb7-a4ce053cdb6a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:00.796315 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ae113-7cdf-4a0f-bdb7-a4ce053cdb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:00.815509 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-008ae113-7cdf-4a0f-bdb7-a4ce053cdb6a None None] MFA Rules not processed for user `08ee7fc8c18641b2ac4c1400638d54f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:46:00.816710 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-008ae113-7cdf-4a0f-bdb7-a4ce053cdb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:00.892110 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1410/2842] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:46:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:46:02.138441 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:02.192216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:02.207169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9617cf-759a-4e73-83f2-a112362c617a service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:02.213545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:02.214500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:02.214785 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:02.215069 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] PATH_INFO: `/v3/users/4adfedbe1b074c1b96f10c3a9d879ab1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:02.221831 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adfedbe1b074c1b96f10c3a9d879ab1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:02.224401 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:02.274415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adfedbe1b074c1b96f10c3a9d879ab1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.283370 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adfedbe1b074c1b96f10c3a9d879ab1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.283966 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adfedbe1b074c1b96f10c3a9d879ab1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.291071 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adfedbe1b074c1b96f10c3a9d879ab1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.291708 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adfedbe1b074c1b96f10c3a9d879ab1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.310167 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1433/2843] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:46:02 2022] DELETE /identity/v3/users/4adfedbe1b074c1b96f10c3a9d879ab1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:46:02.315018 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:02.315519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:02.315653 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:02.343061 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:46:02.344919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:02.376370 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:02.415483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1411/2844] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:46:02.426169 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:02.427711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:02.489573 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee86e66-d4fc-4550-8a9e-544daba6eca5 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:02.499023 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9642c0-bd81-4bb8-b571-f0b3910863a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:02.499911 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9642c0-bd81-4bb8-b571-f0b3910863a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:02.500213 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9642c0-bd81-4bb8-b571-f0b3910863a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:02.500555 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9642c0-bd81-4bb8-b571-f0b3910863a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:02.512695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9642c0-bd81-4bb8-b571-f0b3910863a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:02.514173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9642c0-bd81-4bb8-b571-f0b3910863a4 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:02.524495 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1434/2845] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:46:02.851546 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:02.853759 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:02.875681 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:02.876361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:02.877265 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:02.878189 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcc2f6-7fe0-4590-bba1-3a26fd74e21c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:02.901875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:02.902552 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:02.902827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:02.903275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] PATH_INFO: `/v3/projects/e95b82558df841cab6e144f21099ea6f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:02.904507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95b82558df841cab6e144f21099ea6f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:02.906568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:02.912868 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:46:02.916100 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:02.976788 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95b82558df841cab6e144f21099ea6f', 'request_id': 'req-d843f895-76e3-4614-9da7-1b70cfeb85d5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.981101 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1435/2846] 10.222.0.49 () {58 vars in 1009 bytes} [Sun Jul 31 18:46:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 18:46:02.983707 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95b82558df841cab6e144f21099ea6f', 'request_id': 'req-d843f895-76e3-4614-9da7-1b70cfeb85d5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.990108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95b82558df841cab6e144f21099ea6f', 'request_id': 'req-d843f895-76e3-4614-9da7-1b70cfeb85d5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:02.993182 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:02.994502 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:03.005780 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] The token cache is being invalidate because project e95b82558df841cab6e144f21099ea6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:46:03.006079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:03.006724 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d843f895-76e3-4614-9da7-1b70cfeb85d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:46:03.009264 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1412/2847] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:46:02 2022] DELETE /identity/v3/projects/e95b82558df841cab6e144f21099ea6f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:46:03.013692 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:03.085769 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7612e1-2163-4ef1-83b2-37741a9a55aa None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:03.167286 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:03.168958 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:03.169465 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:03.169884 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:03.172226 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:03.247059 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:03.248956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:03.259837 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1436/2848] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:46:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 18:46:13.002860 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:13.256221 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=73 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:14.252168 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:14.252660 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:14.252835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:14.272779 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] MFA Rules not processed for user `64d0451d06294c968712f411f90f224e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:46:14.274005 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:14.331151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1413/2849] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:46:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:46:24.324265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:31.001273 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:31.003119 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:31.034900 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:31.036485 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:31.060888 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c832995-5010-4ab0-a853-d57fc93317c7 service swift] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:31.064324 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:31.064984 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:31.065258 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:31.065518 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:31.067261 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:31.097698 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b3dd8-27f0-485b-9953-a4c799e19694 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:31.101394 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:31.102015 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:31.102292 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:31.102557 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:31.104375 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:31.130427 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:31.131854 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:31.142048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1437/2850] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:46:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:46:31.172001 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:31.173629 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:31.184152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1414/2851] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:46:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:46:32.740272 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:46:32.741926 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:32.823215 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579627f-7593-4f52-b76a-57b049ab047f service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:46:32.829195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:46:32.829902 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:46:32.830929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:46:32.831282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:46:32.833709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:46:32.849754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:46:32.851350 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:46:32.865489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1438/2852] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:46:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:46:41.180125 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=50 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:46:42.858751 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:47:43.076054 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:43.085183 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcf9eb-1958-49fb-b874-aefe62cadbb4 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:43.089073 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:43.089675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:43.089941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:43.090211 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:43.092051 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:43.105538 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:43.106957 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:43.118218 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1415/2853] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:47:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:47:43.650410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:43.660554 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06299-71c8-48e7-bda1-1280f14ebe24 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:43.664225 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:43.664872 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:43.665149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:43.665415 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:43.667189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:43.680001 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:43.682238 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564c937-b78f-4276-a98d-01f7d82f6151 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:43.705020 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1439/2854] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:47:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:47:45.991156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:45.993398 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:46.053600 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34553a85-9628-40be-a9f0-80805b2b148f service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:46.057719 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20341bc6269f477c952c25437ab7a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:46.058414 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:46.058840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:46.058980 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:46.062202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:46.081504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:46.082938 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:46.094820 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1416/2855] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:47:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:47:51.026488 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:51.026945 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:51.027051 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:51.076878 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] MFA Rules not processed for user `4242cff2780f478099110d0586ad67b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:47:51.078127 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:51.095439 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1440/2856] 10.222.0.49 () {58 vars in 967 bytes} [Sun Jul 31 18:47:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:47:51.136388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:51.137955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:51.184914 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95d3ba-234e-4555-b61b-40000e71bd75 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:51.193615 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64871a43e564c9bbf7ebfdbd512a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:51.194498 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:51.194874 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:51.195238 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:51.210594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:51.210594 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:51.222451 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1417/2857] 10.222.0.49 () {60 vars in 1384 bytes} [Sun Jul 31 18:47:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 18:47:57.781267 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:57.782922 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:57.859775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33ee7d-a29e-4137-b51a-1ad42f360e59 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:57.864744 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:57.865432 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:57.865726 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:57.865997 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] PATH_INFO: `/v3/users/08ee7fc8c18641b2ac4c1400638d54f1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:57.867176 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ee7fc8c18641b2ac4c1400638d54f1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:57.869208 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:57.916155 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ee7fc8c18641b2ac4c1400638d54f1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:57.926510 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ee7fc8c18641b2ac4c1400638d54f1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:57.927179 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ee7fc8c18641b2ac4c1400638d54f1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:57.933993 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ee7fc8c18641b2ac4c1400638d54f1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:57.934662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ee7fc8c18641b2ac4c1400638d54f1'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:57.951515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1441/2858] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:47:57 2022] DELETE /identity/v3/users/08ee7fc8c18641b2ac4c1400638d54f1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 18:47:58.159346 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:58.171436 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1743c2-4b1a-4a91-91e1-860911054370 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:58.232584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:58.233305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:58.233584 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:58.233857 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] PATH_INFO: `/v3/projects/897eef74408f4973bfb541b7f92365c7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:58.235083 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897eef74408f4973bfb541b7f92365c7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:58.237039 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:58.288308 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897eef74408f4973bfb541b7f92365c7', 'request_id': 'req-82a74e88-1004-421b-ae79-d423d2b48442'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.294733 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897eef74408f4973bfb541b7f92365c7', 'request_id': 'req-82a74e88-1004-421b-ae79-d423d2b48442'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.299274 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:47:58.302150 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897eef74408f4973bfb541b7f92365c7', 'request_id': 'req-82a74e88-1004-421b-ae79-d423d2b48442'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.316426 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] The token cache is being invalidate because project 897eef74408f4973bfb541b7f92365c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:47:58.316704 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.317324 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.318879 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1418/2859] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:47:58 2022] DELETE /identity/v3/projects/897eef74408f4973bfb541b7f92365c7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:47:58.323200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:58.324666 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:58.338433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929a90-792c-4112-afa8-45d0ee5f30a0 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:58.363475 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:47:58.401493 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:58.402137 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:58.402411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:58.402678 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] PATH_INFO: `/v3/users/9b8544f860e24a838e7f417f5f4df3ff` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:58.403903 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8544f860e24a838e7f417f5f4df3ff)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:58.405885 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:58.457244 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8544f860e24a838e7f417f5f4df3ff'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.466547 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8544f860e24a838e7f417f5f4df3ff'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.467236 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8544f860e24a838e7f417f5f4df3ff'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.474489 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8544f860e24a838e7f417f5f4df3ff'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.475110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-f622c78e-2778-4776-b089-c2b0b4860147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8544f860e24a838e7f417f5f4df3ff'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.493311 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1442/2860] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:47:58 2022] DELETE /identity/v3/users/9b8544f860e24a838e7f417f5f4df3ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:47:58.679922 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:58.681610 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:58.697656 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a74e88-1004-421b-ae79-d423d2b48442 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:58.765847 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:58.766491 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:58.766766 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:58.767034 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] PATH_INFO: `/v3/projects/944889ed3b654f6e8c390f9fe58b1855` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:58.768247 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944889ed3b654f6e8c390f9fe58b1855)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:58.770341 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:58.818905 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944889ed3b654f6e8c390f9fe58b1855', 'request_id': 'req-7fb81134-d22a-4937-9605-c579dcbd8a79'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.824776 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944889ed3b654f6e8c390f9fe58b1855', 'request_id': 'req-7fb81134-d22a-4937-9605-c579dcbd8a79'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.830322 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944889ed3b654f6e8c390f9fe58b1855', 'request_id': 'req-7fb81134-d22a-4937-9605-c579dcbd8a79'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.846729 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] The token cache is being invalidate because project 944889ed3b654f6e8c390f9fe58b1855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:47:58.847117 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.848132 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7fb81134-d22a-4937-9605-c579dcbd8a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:47:58.850574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1419/2861] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:47:58 2022] DELETE /identity/v3/projects/944889ed3b654f6e8c390f9fe58b1855 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:47:58.857604 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:58.858113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:58.858113 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:58.878840 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:47:58.880111 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:58.944099 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1443/2862] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:47:58.953010 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:58.953296 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:58.953569 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:58.973598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:47:58.974812 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:58.984133 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1420/2863] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:47:58.988795 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:58.998722 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76b9e1-26d0-4e78-ab69-4903e1d828b8 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.004856 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.005474 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.005755 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.006012 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] PATH_INFO: `/v3/domains` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.006721 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80202) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 18:47:59.007011 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.008643 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.021356 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1444/2864] 10.222.0.49 () {58 vars in 1153 bytes} [Sun Jul 31 18:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 18:47:59.025750 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:59.035156 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcdfce-05eb-4e44-ba19-052157065054 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.041152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.041752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.042024 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.042286 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.042942 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.044875 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.089193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1421/2865] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:47:59 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:47:59.094857 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:59.105410 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db0c4b-7450-426e-8c6a-99b5934ca756 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.136871 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:47:59.166662 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.167288 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.167592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.167866 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.168522 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.170517 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.193711 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1445/2866] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:47:59 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:47:59.197515 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:59.208412 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195d150-1df8-4403-b588-680f7d9d3d78 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.213896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.214484 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.214761 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.215030 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.215712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.217376 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.228058 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1422/2867] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:47:59.232284 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:59.242913 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccc169-4cb5-4285-88b5-928574cbaefc admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.248234 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878aa154-8b95-414a-a916-cac4d688cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.248910 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878aa154-8b95-414a-a916-cac4d688cfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.249163 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878aa154-8b95-414a-a916-cac4d688cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.249459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878aa154-8b95-414a-a916-cac4d688cfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.250152 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878aa154-8b95-414a-a916-cac4d688cfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.252116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878aa154-8b95-414a-a916-cac4d688cfcf admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.262185 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1446/2868] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:47:59.266290 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:59.276666 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21420bb-ad37-4cc5-a977-743a94ec1559 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.281930 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.282606 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.283038 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.283284 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] PATH_INFO: `/v3/projects/0bef4721a1fe41d8a8ec8013079a9a93/users/1159df549a4b43a9958f389801be2272/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.298369 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bef4721a1fe41d8a8ec8013079a9a93, user_id=1159df549a4b43a9958f389801be2272, role_id=c6e1e57d3cdf4c9ab14953c6bc8d4436)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.301074 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.316461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1423/2869] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:47:59 2022] PUT /identity/v3/projects/0bef4721a1fe41d8a8ec8013079a9a93/users/1159df549a4b43a9958f389801be2272/roles/c6e1e57d3cdf4c9ab14953c6bc8d4436 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:47:59.321891 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d7b7-5276-4e3e-a6bf-baf968556944 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.322216 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d7b7-5276-4e3e-a6bf-baf968556944 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.322528 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d7b7-5276-4e3e-a6bf-baf968556944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.343321 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-28e4d7b7-5276-4e3e-a6bf-baf968556944 None None] MFA Rules not processed for user `a62909c025a448a2a69b126becb412ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:47:59.344461 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-28e4d7b7-5276-4e3e-a6bf-baf968556944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:59.405620 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1447/2870] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:47:59.415454 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:47:59.417275 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:47:59.423953 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:47:59.476571 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62380d-0988-4353-a9d8-4865fa4647dc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:47:59.480431 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:47:59.481138 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:47:59.481428 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:47:59.481657 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:47:59.490708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:47:59.492141 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:47:59.501746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1424/2871] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:03.462242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.462754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.462980 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.507022 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.auth.core [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] MFA Rules not processed for user `1159df549a4b43a9958f389801be2272`. Rule list: `[]` (Enabled: `True`). {{(pid=80202) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:48:03.508417 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:03.594484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1448/2872] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:48:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:48:03.598896 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:03.610475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a4712-0c14-4f2a-8ebd-b62382989183 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:03.615463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:03.616086 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.616360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.616625 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.617331 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:03.619228 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:03.657067 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1425/2873] 10.222.0.49 () {60 vars in 1153 bytes} [Sun Jul 31 18:48:03 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 18:48:03.658592 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:03.670956 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d20cb-be4d-4632-be78-58232a1adef1 None None] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:03.718269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:48:03.744081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:03.744724 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.745035 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.745309 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.745988 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:03.747950 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:03.776367 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1449/2874] 10.222.0.49 () {60 vars in 1144 bytes} [Sun Jul 31 18:48:03 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 18:48:03.778266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:03.790224 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c079a-4625-41e5-ab56-c363f364fc94 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:03.795686 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:03.796439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.796804 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.797149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.798164 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:03.799623 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:03.814754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1426/2875] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:48:03.819583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:03.830924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570098fb-5d34-43d2-b76b-308f3a6b0a84 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:03.835842 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:03.836524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.836824 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.837097 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] PATH_INFO: `/v3/projects/955cfeb0e4694584a723012812bee17d/users/866b0ca69dfb4a02bddadcc604cf463a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.853610 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955cfeb0e4694584a723012812bee17d, user_id=866b0ca69dfb4a02bddadcc604cf463a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:03.856455 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:03.876932 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1450/2876] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:48:03 2022] PUT /identity/v3/projects/955cfeb0e4694584a723012812bee17d/users/866b0ca69dfb4a02bddadcc604cf463a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:48:03.877926 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:03.888535 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf53fd-14b6-476f-ae2f-9e6d85aa2688 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:03.957266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b446af-d34b-482d-8411-49cef45911df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:03.957931 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b446af-d34b-482d-8411-49cef45911df admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.958207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b446af-d34b-482d-8411-49cef45911df admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.958475 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b446af-d34b-482d-8411-49cef45911df admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.959170 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b446af-d34b-482d-8411-49cef45911df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:03.960794 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b446af-d34b-482d-8411-49cef45911df admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:03.970882 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1427/2877] 10.222.0.49 () {58 vars in 1122 bytes} [Sun Jul 31 18:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 18:48:03.975257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:03.986741 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206e4d-c433-422f-a1b2-3a10fee419f1 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:03.991562 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:03.992233 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:03.992496 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:03.992819 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] PATH_INFO: `/v3/projects/955cfeb0e4694584a723012812bee17d/users/866b0ca69dfb4a02bddadcc604cf463a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:03.994728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955cfeb0e4694584a723012812bee17d, user_id=866b0ca69dfb4a02bddadcc604cf463a, role_id=e4e7885ec8ed4564a31d2cb9dfae8fb1)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:03.997411 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:04.008433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1451/2878] 10.222.0.49 () {60 vars in 1483 bytes} [Sun Jul 31 18:48:03 2022] PUT /identity/v3/projects/955cfeb0e4694584a723012812bee17d/users/866b0ca69dfb4a02bddadcc604cf463a/roles/e4e7885ec8ed4564a31d2cb9dfae8fb1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:48:08.415709 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c008-300f-460a-886d-68f62d87e4d7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:08.416207 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c008-300f-460a-886d-68f62d87e4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:08.416401 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c008-300f-460a-886d-68f62d87e4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:08.447478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-be09c008-300f-460a-886d-68f62d87e4d7 None None] MFA Rules not processed for user `866b0ca69dfb4a02bddadcc604cf463a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:48:08.448865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be09c008-300f-460a-886d-68f62d87e4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:08.521567 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:48:08.532754 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1428/2879] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:48:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:48:08.549357 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:08.551800 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:08.630008 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a78eb-d712-40dd-9a50-2e188cb3fa70 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:08.635480 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:08.636114 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:08.636384 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:08.636648 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:08.694593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:08.696122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:08.705658 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1452/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:48:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:09.716700 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:09.717244 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:09.717360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:09.737840 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] MFA Rules not processed for user `1159df549a4b43a9958f389801be2272`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:48:09.739081 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:09.749233 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1429/2881] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:48:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:48:18.701419 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:48:19.731899 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:48:24.318636 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:24.320260 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:24.380301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ca309-d502-463e-a06e-c6cafcd06019 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:24.384242 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:24.384858 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:24.385132 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:24.385396 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:24.387434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:24.397340 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:24.398746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:24.406940 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1453/2882] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:48:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:25.906492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:25.907933 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:25.981040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad4226-19ef-49df-969f-6414529f7866 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:25.986301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:25.986927 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:25.987202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:25.987469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:25.989411 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:26.001319 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:26.002901 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46744df0-8ba3-47cd-9f6e-313eb9e884f0 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:26.013135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1430/2883] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:48:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:28.648594 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:28.650334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:28.669079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cd2c7-b417-4392-b38a-0e7703510708 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:28.676524 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:28.677274 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:28.677616 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:28.677951 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:28.680143 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:28.693516 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:28.696576 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:28.706473 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1454/2884] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:48:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:33.892348 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:33.892870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:33.892870 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:33.912897 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] MFA Rules not processed for user `64d0451d06294c968712f411f90f224e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:48:33.914114 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:33.974096 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1431/2885] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:48:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:48:38.701618 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=32 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:48:39.074634 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:39.087149 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b09926-5da2-4d38-a819-3760eb25c1dd service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:39.092996 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:39.093746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:39.094090 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:39.094426 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:39.096459 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:48:39.110407 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:39.112124 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:39.124361 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1455/2886] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:48:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:41.634814 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:48:41.647380 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e03b2-b86c-4762-9a47-692e1a217e9b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:48:41.651577 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64957b98-08ee-492f-a863-61a63187c363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:48:41.652208 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64957b98-08ee-492f-a863-61a63187c363 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:48:41.652482 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64957b98-08ee-492f-a863-61a63187c363 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:48:41.652865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64957b98-08ee-492f-a863-61a63187c363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:48:41.663080 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64957b98-08ee-492f-a863-61a63187c363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:48:41.664519 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64957b98-08ee-492f-a863-61a63187c363 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:48:41.674592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1432/2887] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:48:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:48:49.119738 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:48:51.670160 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=63 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:50:24.931220 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:50:24.943138 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a172442-2c33-4558-8e14-bed0695531b1 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:50:24.949122 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:50:24.949756 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:50:24.950026 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:50:24.950292 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:50:24.952319 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:50:24.963906 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:50:24.965506 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:50:24.975606 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1456/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:50:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:50:34.971508 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:50:45.985237 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:50:45.985762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:50:45.985762 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:50:46.006465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] MFA Rules not processed for user `64d0451d06294c968712f411f90f224e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 18:50:46.007678 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:50:46.015920 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1433/2889] 10.222.0.49 () {58 vars in 958 bytes} [Sun Jul 31 18:50:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 18:50:55.379977 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:50:55.389718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6d2e1-aeb8-4562-8e58-13070903b114 service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:50:55.393626 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:50:55.394227 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:50:55.394499 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:50:55.394754 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:50:55.396500 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:50:55.405964 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:50:55.407391 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:50:55.415515 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1457/2890] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:50:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:50:55.538504 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:50:55.549420 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3843ecd-820f-4867-a30c-80869b3aafff None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:50:55.553418 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:50:55.554163 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:50:55.554439 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:50:55.554703 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:50:55.556635 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:50:55.567040 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:50:55.568469 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:50:55.577066 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1434/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:50:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:50:55.660838 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:50:55.669846 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971732cb-7e6d-4748-8382-cc10417c33f5 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:50:55.673434 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:50:55.674038 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:50:55.674310 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:50:55.674575 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:50:55.676279 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:50:55.689700 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:50:55.691040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:50:55.701718 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1458/2892] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:50:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:50:58.061943 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:50:58.076004 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6241ba-2bbe-4e69-b4fc-1c5211edf926 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:50:58.082402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:50:58.083479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:50:58.083827 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:50:58.084457 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:50:58.087011 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:50:58.097841 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:50:58.099405 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:50:58.108031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1435/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:50:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:51:05.697253 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=12 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:51:08.104149 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=15 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:20.581275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:20.595195 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d77eb1-810d-43b8-a9ce-04ead06f1c62 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:20.601282 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9958737194be4488a1e09e6e48557d75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:20.601960 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:20.602380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:20.602791 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:20.604602 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:20.618389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:20.619981 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:20.632719 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1459/2894] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:52:27.763528 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:27.765403 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:27.782381 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead07a53-9ee9-46e8-9e3f-3bc1f48af2b4 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:27.788130 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:27.788784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:27.789063 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:27.789330 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:27.791265 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:27.806461 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:27.808179 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:27.818676 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1436/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:52:30.627463 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=6 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:31.043728 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:31.109483 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11394473-9a95-4c53-a271-92f87906d829 service nova] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:31.114599 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:31.115269 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:31.115545 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:31.115818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] PATH_INFO: `/v3/users/1159df549a4b43a9958f389801be2272` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:31.117040 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1159df549a4b43a9958f389801be2272)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:31.119079 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:31.174275 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1159df549a4b43a9958f389801be2272'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.183257 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1159df549a4b43a9958f389801be2272'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.183917 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1159df549a4b43a9958f389801be2272'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.191128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1159df549a4b43a9958f389801be2272'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.191929 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1159df549a4b43a9958f389801be2272'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.210397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1460/2896] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:52:31 2022] DELETE /identity/v3/users/1159df549a4b43a9958f389801be2272 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:31.398568 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:31.410352 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e6ecb-0148-4273-ae8d-879d97e2214c service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:31.467193 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:31.467838 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:31.468108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:31.468382 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] PATH_INFO: `/v3/projects/0bef4721a1fe41d8a8ec8013079a9a93` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:31.469554 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bef4721a1fe41d8a8ec8013079a9a93)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:31.471463 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:31.525135 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bef4721a1fe41d8a8ec8013079a9a93', 'request_id': 'req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.532031 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bef4721a1fe41d8a8ec8013079a9a93', 'request_id': 'req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.537784 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bef4721a1fe41d8a8ec8013079a9a93', 'request_id': 'req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.551937 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] The token cache is being invalidate because project 0bef4721a1fe41d8a8ec8013079a9a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:52:31.552222 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.552865 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.554388 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1437/2897] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:52:31 2022] DELETE /identity/v3/projects/0bef4721a1fe41d8a8ec8013079a9a93 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:31.558730 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:31.560243 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:31.573859 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e49d26-9351-4655-9025-f850e5e0d72b admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:31.636006 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:31.636654 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:31.636914 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:31.637189 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] PATH_INFO: `/v3/users/866b0ca69dfb4a02bddadcc604cf463a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:31.638397 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=866b0ca69dfb4a02bddadcc604cf463a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:31.640343 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:31.663068 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:31.691412 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866b0ca69dfb4a02bddadcc604cf463a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.701116 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866b0ca69dfb4a02bddadcc604cf463a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.701962 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866b0ca69dfb4a02bddadcc604cf463a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.709892 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866b0ca69dfb4a02bddadcc604cf463a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.710570 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866b0ca69dfb4a02bddadcc604cf463a'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:31.729818 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1461/2898] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:52:31 2022] DELETE /identity/v3/users/866b0ca69dfb4a02bddadcc604cf463a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:31.915432 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:31.916955 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:31.929892 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b68d4b-95c4-49b7-aea2-1123e0ce4939 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:32.004717 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:32.005402 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:32.005675 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:32.005941 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] PATH_INFO: `/v3/projects/955cfeb0e4694584a723012812bee17d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:32.007092 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] RBAC: Authorizing `identity:delete_project(project_id=955cfeb0e4694584a723012812bee17d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:32.009108 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:32.029793 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:32.057799 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955cfeb0e4694584a723012812bee17d', 'request_id': 'req-a5145319-d7fd-4a7e-8131-8ca2e73576be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:32.066496 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955cfeb0e4694584a723012812bee17d', 'request_id': 'req-a5145319-d7fd-4a7e-8131-8ca2e73576be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:32.073448 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955cfeb0e4694584a723012812bee17d', 'request_id': 'req-a5145319-d7fd-4a7e-8131-8ca2e73576be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:32.088223 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] The token cache is being invalidate because project 955cfeb0e4694584a723012812bee17d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:52:32.088478 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:32.089175 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:32.090752 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1438/2899] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:52:31 2022] DELETE /identity/v3/projects/955cfeb0e4694584a723012812bee17d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:41.726129 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:42.084919 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:45.014564 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:45.016644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:45.091083 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a352b9b-d4c0-4858-bb5e-08ec15e9a9a4 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:45.135818 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:45.137453 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:45.162136 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:45.162770 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:45.163778 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:45.164110 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:45.167278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:45.183939 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:45.186039 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:45.196825 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1462/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:52:45.207033 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145319-d7fd-4a7e-8131-8ca2e73576be admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:45.212328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:45.212328 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:45.212592 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:45.212880 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:45.214574 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:45.293634 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:45.295082 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:45.306305 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1439/2901] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 18:52:46.490285 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:46.491775 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:46.502746 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9741ac2-6c14-4e83-b2f4-517944a85013 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:46.507389 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:46.508347 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:46.508695 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:46.509096 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:46.512081 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:46.533979 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:46.535403 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:46.546353 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1463/2902] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:52:50.739266 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:50.749580 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e56435-26d6-4276-906e-b3c36bcc75ec service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:50.754507 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f590f668ad724bf087b164bcf3f543ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:50.755152 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:50.755423 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:50.755693 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:50.757659 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:50.829954 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:50.831748 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:50.842479 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1440/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:52:53.911380 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:53.913644 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:53.996661 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78e017-baf1-4fdf-aeaf-be04e9d047f3 service neutron] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:54.002123 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:54.002771 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] REQUEST_METHOD: `GET` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:54.003048 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:54.003317 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:54.005109 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:54.016709 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:54.018314 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:54.028584 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1464/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:52:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:52:57.202079 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:57.203746 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:57.216671 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e7679-6d27-4f88-8d3f-8032d2b26bf5 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:57.225465 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:57.225908 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:57.226342 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:57.226835 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] PATH_INFO: `/v3/users/64d0451d06294c968712f411f90f224e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:57.228492 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d0451d06294c968712f411f90f224e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:57.230769 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:57.274878 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:57.290298 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d0451d06294c968712f411f90f224e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.297825 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d0451d06294c968712f411f90f224e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.298582 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d0451d06294c968712f411f90f224e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.310483 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d0451d06294c968712f411f90f224e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.317486 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d0451d06294c968712f411f90f224e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.323708 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1441/2905] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:52:57 2022] DELETE /identity/v3/users/64d0451d06294c968712f411f90f224e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:57.515501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:57.526485 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e842f-9ca6-474b-9ad2-215bb27e158e service placement] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:57.560202 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=100 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:57.586924 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:57.587593 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:57.587865 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:57.588128 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] PATH_INFO: `/v3/projects/11018257977c4b8ba8f008c0c6ae65fd` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:57.589301 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11018257977c4b8ba8f008c0c6ae65fd)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:57.591193 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:57.638278 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11018257977c4b8ba8f008c0c6ae65fd', 'request_id': 'req-10e857e8-b8bd-4f65-8323-571b870d7656'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.644583 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11018257977c4b8ba8f008c0c6ae65fd', 'request_id': 'req-10e857e8-b8bd-4f65-8323-571b870d7656'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.650809 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11018257977c4b8ba8f008c0c6ae65fd', 'request_id': 'req-10e857e8-b8bd-4f65-8323-571b870d7656'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.665557 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] The token cache is being invalidate because project 11018257977c4b8ba8f008c0c6ae65fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:52:57.665847 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.666484 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.668320 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1465/2906] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:52:57 2022] DELETE /identity/v3/projects/11018257977c4b8ba8f008c0c6ae65fd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:57.672301 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:57.674022 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:57.689599 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713afc67-4614-4a63-a1db-5c4e471bcdec admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:57.758778 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:57.759438 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:57.759712 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:57.759973 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] PATH_INFO: `/v3/users/df9dfec58ce844659fd0d79d684c0d2d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:57.761200 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9dfec58ce844659fd0d79d684c0d2d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:57.763151 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:57.811757 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9dfec58ce844659fd0d79d684c0d2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.820148 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9dfec58ce844659fd0d79d684c0d2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.820751 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9dfec58ce844659fd0d79d684c0d2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.832326 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9dfec58ce844659fd0d79d684c0d2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.838996 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9dfec58ce844659fd0d79d684c0d2d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:57.844684 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1442/2907] 10.222.0.49 () {58 vars in 1224 bytes} [Sun Jul 31 18:52:57 2022] DELETE /identity/v3/users/df9dfec58ce844659fd0d79d684c0d2d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:52:58.025334 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Authenticating user token {{(pid=80202) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:52:58.026901 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.fernet_utils [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80202) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:52:58.041053 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e857e8-b8bd-4f65-8323-571b870d7656 admin admin] Validating token access rules against request {{(pid=80202) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:52:58.097173 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62909c025a448a2a69b126becb412ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57e06c47f85e41698c0742550f1a32ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80202) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:52:58.097876 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:52:58.098133 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:52:58.098400 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] PATH_INFO: `/v3/projects/c403eb8800ad464d9fc2c7abf461699a` {{(pid=80202) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:52:58.099688 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c403eb8800ad464d9fc2c7abf461699a)` {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:52:58.101664 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] RBAC: Authorization granted {{(pid=80202) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:52:58.163200 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c403eb8800ad464d9fc2c7abf461699a', 'request_id': 'req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:58.169531 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c403eb8800ad464d9fc2c7abf461699a', 'request_id': 'req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:58.175501 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c403eb8800ad464d9fc2c7abf461699a', 'request_id': 'req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8'} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:58.179230 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:52:58.191217 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] The token cache is being invalidate because project c403eb8800ad464d9fc2c7abf461699a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80202) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 18:52:58.191433 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:58.192088 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG keystone.notifications [None req-ce3bc0f6-56c0-4a26-af85-f36beeb20fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80202) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 18:52:58.193660 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: [pid: 80202|app: 0|req: 1466/2908] 10.222.0.49 () {58 vars in 1233 bytes} [Sun Jul 31 18:52:58 2022] DELETE /identity/v3/projects/c403eb8800ad464d9fc2c7abf461699a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 18:53:07.841459 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=71,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:53:08.190456 dv-cinder-849846-5-iscsi devstack@keystone.service[80202]: DEBUG dbcounter [-] [80202] Writing DB stats keystone:SELECT=2 {{(pid=80202) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 18:55:27.969456 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 18:55:27.971202 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:55:28.130360 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3af68-c570-49e1-9fcb-46939151f7c5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 18:55:28.199598 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5686862728f34d64af248e12358541d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf16437efd0f4e3188059acaacd8a01a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 18:55:28.205143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 18:55:28.205143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 18:55:28.205143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 18:55:28.205143 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 18:55:28.215576 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 18:55:28.217234 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5efba8-9708-403c-ac55-00c1fa527898 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 18:55:28.227800 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1443/2909] 10.222.0.49 () {60 vars in 1365 bytes} [Sun Jul 31 18:55:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 18:55:38.223363 dv-cinder-849846-5-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}